text:
GCP-Plugin Donation

Motivation

The GCP Audit Logs Plugin's primary purpose is to detect security threats, vulnerabilities, and compliance risks by analyzing the ingested GCP audit logs. The default security detection rules were built with the MITRE & ATT&CK framework in mind, which provides a comprehensive and industry-standard way to identify and classify different types of security threats.
The GCP Audit Logs Plugin can help security teams identify and respond to security incidents quickly, improve compliance posture, and reduce overall risk to the organization. It provides a comprehensive and centralized view of security events across multiple GCP services and can help detect and prevent unauthorized access, data exfiltration, and other types of malicious activity.
Feature

By leveraging GCP audit logs, the GCP Audit Logs Plugin provides deep insights into the activities of different users, services, and resources in your GCP environment. The GCP Audit Logs Plugin's advanced ebpf capabilities enable it to identify anomalous activities and raise alerts when it detects suspicious or malicious behavior.
The GCP Audit Logs Plugin also offers customizable detection rules that enable you to fine-tune the detection capabilities to suit your organization's specific needs. You can customize the rules to detect specific types of security threats, monitor specific users or services, and track specific resources or data types.
Additional context
The GCP Audit Logs Plugin comes with pre-built security detection rules designed to detect security threats based on the MITRE & ATT&CK framework. These rules are constantly updated to ensure that the security agent is always detecting the latest threats and vulnerabilities.
The default security detection rules cover the following areas:
Identity and Access Management (IAM)
Network Security
Data Security
Compliance
Infrastructure Security
Cloud Service Providers
The GCP Audit Logs Plugin's detection rules can identify threats such as:
Privilege escalation
Unauthorized access
Data exfiltration
Denial of Service (DoS) attacks
Insider threats
Suspicious network activity

I followed you so far, and it's a big +1 from me! Great project and super valuable contribution for our project! 🚀
cc @falcosecurity/plugins-maintainers

@jasondellaluce Thank you so much for your support!

Big +1 for me!
Thank you!

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

/close

@jasondellaluce: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

authors: ahmedameenaim jasondellaluce leogr poiana

repo: falcosecurity/plugins


text:
Stop JiJ'ing Trinkets

Due to the size of the Trinkets jar, and to reduce end-user confusion, I've been asked to stop including Trinkets in Scarves and just list it as a dependency. This is reasonable! I'm leaving a note here so I don't forget when I get to updates.

While at that, seems like a redundant include on jankson you could slim down too

source: gharchive/issue

authors: falkreon fzzyhmstrs

repo: falkreon/Scarves

license: MIT

license_type: permissive

license_source: github-api


text:
v2.2.0: new source of data has to be introduced in major version bump, since it is breaking change

 "dependencies": {
    "country-list": "^2.1.0", // resolves to 2.2.0
  },

My packages that rely on this one was kinda broken since new data source change the mapping for getting country codes. Example, previously for "United States" the correct country code was returned, in 2.2.0 it has to be "United States of America", otherwise it will be undefined.
I do recommend to remove this version and create a new one by bumping a major number, 3.0.0
Please, see this https://semver.org/

Hi,
actually this release should have been a patch according to semver since it only fixes bugs.
And one of the bugs was that it accepted "United States" as valid input when it shouldn't have.
Using semver for a lib that is essentially a database is hard.
But I understand how this affects you and unfortunately the only solution I see is to use the overwrite() functionality.
const { overwrite, getCode } = require('country-list');
overwrite([{
  code: 'US',
  name: 'United States'
}])

console.log(getCode('United States')); // US

If you need to accept different spellings for country names I suggest looking into +
https://www.npmjs.com/package/country-mapper

authors: bohdanrutylo fannarsh

repo: fannarsh/country-list

license: mit

license_source: bigquery


text:
Equivalent definitions

Certain typeclasses can be satisfied using one of several equivalent minimal definitions. For example Traversable, which can be satisfied by defining either of sequence or traverse, or Bifunctor, which can be satisfied by defining either of bimap or lmap + rmap.
It is necessary that for each equivalent minimal definition, there exist universally applicable derivations of all the other definitions. As a practical example, it is possible to define both lmap and rmap given a definition of bimap:
const deriveLMapAndRMap = ({ bimap }) => ({
  lmap: f => bimap(f)(x => x),
  rmap: f => bimap(x => x)(f)
})

Similarly, it is possible to derive bimap given both lmap and rmap:
const deriveBimap = ({ lmap, rmap }) => ({
  bimap: f => g => compose(lmap(f))(rmap(g))
})

Hence the two definitions are totally isomorphic, and either should do for defining Bifunctor. While this case is rather trivial, there are other classes where the equivalence between minimal defs may have significant differences. For example, fantasy-land requires users to define reduce to implement Foldable, whereas it may be more convenient or performant to define foldMap for certain kinds of foldables.
An implementer of a class instance for which there are multiple equivalent definitions should have the flexibility to satisfy any one of the minimal definitions (for reasons of clarity/performance), and claim compliance with the spec. The spec should specify derivations of common minimal definitions from each other (as part of the laws).

This concept extends past equivalent definitions within  a single class. Certain typeclasses have "intrinsic" superclasses, for which the entire implementation of the superclass instance may simply be obtained for free from the child instance.
For example, Functor and Applicative are "intrinsic" superclasses of Monad, so in practical terms defining a monadic functor instance just comes down to defining of + chain. All the other things (lift2/ap, map) can simply be derived from those two methods. You might also choose to implement the equivalent definition join + map instead of of + chain (and in fact this might have a significant performance impact for certain kinds of monads).
There's many more examples (traverse gives you both map and reduce/foldMap), but I think that illustrates the gist of the problem. In case you're interested in looking through more equivalent definitions, I have several more that I'm using day to day in https://github.com/masaeedu/fp. Here for example are some derivations of map from implementations of the subclasses of Functor.

The fantasy land spec should provide a vocabulary for talking about equivalent minimal definitions of typeclasses, and should accommodate common equivalent definitions of the classes it specifies.

This seems reasonable to me. The derivations could be defined in sanctuary-type-classes so Z.map, for example, could operate on a value which provides fantasy-land/bimap but not fantasy-land/map.
We should consider the pros and cons of the derivations living in sanctuary-type-classes as opposed to in a dedicated derivations package.
Option 1

Author defines an algebraic data type.
Author imports the derivations package and uses it to fill in the gaps.
Author releases a package with all appropriate methods defined.
Users interact with the ADT via functions provided by sanctuary-type-classes or another library.

Option 2

Author defines an algebraic data type.
Author releases a package with the minimal number of methods defined.
Users interact with the ADT via functions provided by sanctuary-type-classes or another library.
These functions use derivations when necessary.

authors: davidchambers masaeedu

repo: fantasyland/fantasy-land


text:
Properly handle null upsert keys

When you naively upsert against a column that might be null, you risk creating duplicate rows (and not updating the data you expect) because null does not equal null.
In the past, we raised an error if a table didn't enforce NOT NULL (or REQUIRED on BigQuery). This prevented people from making mistakes, but it made it very hard to use dbcrossbar against legacy tables (or with data that may contain a negligible fraction of nulls that you want to ignore).
This PR removes the NOT NULL enforcement. It also adds a subquery into the upsert MERGE statement that drops rows that have null upsert keys. This will be the desired behavior 95% of the time.

It also adds a subquery into the upsert MERGE statement that drops rows that have null upsert keys. This will be the desired behavior 95% of the time.

This means that we'd silent dropping data with no error or warning. Could you please explain why that would be a "desired behavior"?

I think what we really need here is a test case, because otherwise it's really hard to think about the ideal behavior.

@emk having written a million ETL jobs at this point, my desired behavior is always

strip whitespace (including newlines)
convert numerics to floats
if i specify a primary key, and 0.5% of the data is missing that primary key, just drop that data

you've talked about a --lossy option before... i can imagine ^^ being separate options... or we could just build the tool we want to use and accept that ^ is what an ETL pro almost certainly wants

having written a million ETL jobs at this point

This may be a disagreement of scope. I agree that good ETL tools are important, but I believe that dbcrossbar should remain a simple data-moving tool with a clear purpose, and that ETL cleanup should be done using either:

Tools like scrubcsv and falconeri to fix CSV files before importing them, or
Native BigQuery SQL to clean up and MERGE the data once imported.

If we start adding ETL and data-cleaning features into dbcrossbar, there's no obvious or clear stopping point. I feel that dbcrossbar's job to move data from point A to point B, with as little change and data loss as humanly possible.
dbcrossbar also has massively less parallelism available than either falconeri or BigQuery, so it's not an efficient place to munge the CSVs.

There is now an option in scrubcsv which can be used to drop rows that are missing a primary key. The official recommendation for handling this problem at scale is to use:

Use scrubcsv --drop-row-if-null=$COL to remove broken rows. If you need to this at cluster scale, use falconeri or a similar tool to distribute the job. This is much more scalable than dbcrossbar could ever be, and it doesn't compromise dbcrossbar's mission to be a dumb-but-fast data pipe.
Use dbcrossbar cp $SOURCE dbcrossbar-schema:- to fetch the source schema, mark any primary key columns as mandatory, and pass the edited --schema when using dbcrossbar to import data into BigQuery.

source: gharchive/pull-request

authors: emk seamusabshere

repo: faradayio/dbcrossbar


text:
Does shuffle=true already exists for learn.fit_one_cycle, and for learn.fit , because there is no parameter for the same , and if not how can we achieve shuffline in our training data during training?

Describe the bug

Provide your installation details

To Reproduce

Expected behavior

Screenshots

Additional context

Any kind of support or suggestions for doing the same are suggested, also monkey patches to already existing functions are also welcome.
Thanking you all in advance,
Harshit


text:
Added 'data' param on model_summary - it was inherited from global scope

Using the exported method model_summary(...) outside of the notebook caused an error because the function uses a global scoped 'data' variable.
New signature
def model_summary(run, learn, data, find_all=False):
..

New usage
model_summary(run, learn, data)

Could you run tools/run-after-git-clone then re-push this, so we don't have the metadata?

Sure!

Is this the tool?
*tools/fastai-nbstripout -d dev_course/dl2/ipynb
I run it and it seems to remove the metadata.
https://docs.fast.ai/dev/develop.html
Running tools/run-after-git-clone didn't change my notebooks.

tools/run-after-git-clone calls that tool for you when you push/diff/etc, FYI.
--
Jeremy Howard
j@howard.fm
On Thu, Apr 18, 2019, at 4:23 PM, artste wrote:

Is this the tool?
*tools/fastai-nbstripout -d dev_course/dl2/ipynb

I run it and it seems to remove the metadata.
https://docs.fast.ai/dev/develop.html

Running tools/run-after-git-clone didn't change my notebooks.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub https://github.com/fastai/fastai_docs/pull/97#issuecomment-484721081, or mute the thread https://github.com/notifications/unsubscribe-auth/AACUW53Y7O6WUKNFOQLMQALPRD7G7ANCNFSM4HG7L5UA.


text:
Fix attribute error in read_checks function

Closes https://github.com/fastai/fastdownload/issues/11
I tried the first example on the fastdownload README page, using the url of one of the standard fastai datasets available on S3. It raised an attribute error, because variable fmod was an empty dictionary instead of the expected pathlib object.
from fastdownload import FastDownload

url = 'https://s3.amazonaws.com/fast-ai-sample/mnist_tiny.tgz'
d = FastDownload()
path = d.get(url)

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-50-389349d26777> in <module>
      3 url = 'https://s3.amazonaws.com/fast-ai-sample/mnist_tiny.tgz'
      4 d = FastDownload()
----> 5 path = d.get(url)

~/anaconda3/envs/napari-dev/lib/python3.9/site-packages/fastdownload/core.py in get(self, url, extract_key, force)
    119             data = self.data_path(extract_key, urldest(url, self.arch_path()))
    120             if data.exists(): return data
--> 121         self.download(url, force=force)
    122         return self.extract(url, extract_key=extract_key, force=force)

~/anaconda3/envs/napari-dev/lib/python3.9/site-packages/fastdownload/core.py in download(self, url, force)
     94         "Download `url` to archive path, unless exists and `self.check` fails and not `force`"
     95         self.arch_path().mkdir(exist_ok=True, parents=True)
---> 96         return download_and_check(url, urldest(url, self.arch_path()), self.module, force)
     97
     98     def rm(self, url, rm_arch=True, rm_data=True, extract_key='data'):

~/anaconda3/envs/napari-dev/lib/python3.9/site-packages/fastdownload/core.py in download_and_check(url, fpath, fmod, force)
     61     "Download `url` to `fpath`, unless exists and `check` fails and not `force`"
     62     if not force and fpath.exists():
---> 63         if check(fmod, url, fpath): return fpath
     64         else: print("Downloading a new version of this dataset...")
     65     res = download_url(url, fpath)

~/anaconda3/envs/napari-dev/lib/python3.9/site-packages/fastdownload/core.py in check(fmod, url, fpath)
     47 def check(fmod, url, fpath):
     48     "Check whether size and hash of `fpath` matches stored data for `url` or data is missing"
---> 49     checks = read_checks(fmod).get(url)
     50     return not checks or path_stats(fpath)==checks
     51

~/anaconda3/envs/napari-dev/lib/python3.9/site-packages/fastdownload/core.py in read_checks(fmod)
     40 def read_checks(fmod):
     41     "Evaluated contents of `download_checks.py`"
---> 42     if not fmod.exists(): return {}
     43     txt = fmod.read_text()
     44     return eval(txt) if txt else {}

AttributeError: 'dict' object has no attribute 'exists'

This PR avoids that error, by checking whether fmod is an empty dictionary before continuing.

I'm not super familiar with nbdev, so if there's anything about the process I should have done differently please let me know.
I looked at both:

the fastdownload contributing guide, and
the nbdev tutorial

...but I don't think I had things working properly (for example, running nbdev_test_nbs on the master branch didn't pass cleanly, which might indicate something is wrong with my setup)

Note: I think this is probably unrelated to https://github.com/fastai/fastdownload/issues/16

I have the same issue

Many thanks @GenevieveBuckley !

@jph00 Just wanted to note that the latest version of fastdownload on PyPI (0.0.5) still has this bug. A new release would be amazing 😄 I'm happy to help where I can.
For interest sake, I'm currently working around it by installing from GitHub (pinned to the latest commit):
pip install git+https://github.com/fastai/fastdownload.git@2ec6563fef7c92ea975871c8cbcea1a0b02917a1

Oh sorry @seeM my bad! Thanks for letting me know. Just pushed a release.

Thank you, very much appreciated. I really love the library! It's such a pleasure to use 😌

authors: GenevieveBuckley jph00 seeM tcapelle

repo: fastai/fastdownload


text:
Master bar not showing in console

The master bar is not showing on console.
bar = master_bar(range(10))
for e in bar:
    for i in progress_bar(range(10), parent=bar):
        sleep(0.05)

On the notebook I get what I want, but not on the terminal.

But on console only get the child bar:

Yes that is expected. You can't flush more than one line so you can't have a nested progress bar in console mode. You'll get one like per epoch instead, this is the intended behavior.

authors: sgugger tcapelle

repo: fastai/fastprogress


text:
Schema generated with Tuple<Union> incompatible to fast-json-stringify

Prerequisites

[X] I have written a descriptive issue title
[X] I have searched existing issues to ensure the bug has not already been reported

Fastify version
4.26.2
Plugin version
4.0.0
Node.js version
20.11.1
Operating system
Linux
Operating system version (i.e. 20.04, 11.3, 10)
Ubuntu 22.04.4
Description
Tuple<Union> leads to serialization error. The code below creates a fastify server with 3 endpoints. Each responds with exactly what it receives. Only the schemas are slightly different:

/array: acceptes  Array<Union>
/unionTuple: accepts  Tuple<Union>
/numTuple: accepts  Tuple<Number>

One can reproduce the error by sending requests to /unionTuple with body [1, 2]. Everything is fine when sending the same request to the other 2 endpoints.
import fastify from 'fastify';
import { type TypeBoxTypeProvider } from '@fastify/type-provider-typebox';
import { Type } from '@sinclair/typebox';

const app = fastify(
  { ajv: { customOptions: { coerceTypes: false } } },
).withTypeProvider<TypeBoxTypeProvider>();

const StringOrNum = Type.Union([Type.String(), Type.Number()]);

const UnionArray = Type.Array(StringOrNum);
const ArraySchema = {
  body: UnionArray,
  response: {
    200: UnionArray,
  },
};
app.post('/array', { schema: ArraySchema }, (req, res) => {
  const array = req.body;
  res.status(200).send(array);
});

const UnionTuple = Type.Tuple([StringOrNum, StringOrNum]);
const UnionTupleSchema = {
  body: UnionTuple,
  response: {
    200: UnionTuple,
  },
};
app.post('/unionTuple', { schema: UnionTupleSchema }, (req, res) => {
  const tuple = req.body;
  res.status(200).send(tuple);
});

const NumTuple = Type.Tuple([Type.Number(), Type.Number()]);
const NumTupleSchema = {
  body: NumTuple,
  response: {
    200: NumTuple,
  },
};
app.post('/numTuple', { schema: NumTupleSchema }, (req, res) => {
  const tuple = req.body;
  res.status(200).send(tuple);
});

app.setErrorHandler((error, req, res) => {
  if ('serialization' in error) {
    res.status(500).send({ 'serialization error': error.message });
  } else {
    res.send(error);
  }
});

app.listen({ port: 3000 });

Link to code that reproduces the bug
No response
Expected Behavior
Type<Union> is expected to serialize the data successfully.
The endpoint have identical schema for request and response. So if data format is invalid, some exception should be thrown upon request validation. But the error I encountered happens during serialization, so I suspect that it must have something to do with fast-json-stringify failing to serialize the response body.

After further investigation, it seems like the generated schema serializer has a bug at the 4th if statement in anonymous0(). Inside this statement is an if-else block. Apparently, it always executes the else block and throw Error('Item at 0 does not match schema definition.'). (because the corresponding if condition is undefined).
(function anonymous(validator, serializer) {
    const JSON_STR_BEGIN_OBJECT = "{";
    const JSON_STR_END_OBJECT = "}";
    const JSON_STR_BEGIN_ARRAY = "[";
    const JSON_STR_END_ARRAY = "]";
    const JSON_STR_COMMA = ",";
    const JSON_STR_COLONS = ":";
    const JSON_STR_QUOTE = '"';
    const JSON_STR_EMPTY_OBJECT = JSON_STR_BEGIN_OBJECT + JSON_STR_END_OBJECT;
    const JSON_STR_EMPTY_ARRAY = JSON_STR_BEGIN_ARRAY + JSON_STR_END_ARRAY;
    const JSON_STR_EMPTY_STRING = JSON_STR_QUOTE + JSON_STR_QUOTE;
    const JSON_STR_NULL = "null";

    function anonymous0(obj) {
        // #

        if (obj === null) return JSON_STR_EMPTY_ARRAY;
        if (!Array.isArray(obj)) {
            throw new TypeError(`The value of '#' does not match schema definition.`);
        }
        const arrayLength = obj.length;

        if (arrayLength > 2) {
            throw new Error(`Item at 2 does not match schema definition.`);
        }

        const arrayEnd = arrayLength - 1;
        let value;
        let json = "";
        value = obj[0];
        if (0 < arrayLength) {
            if (undefined) { // ====== ====== ====== problem here ====== ====== ======
                if (validator.validate("__fjs_root_3#/items/0/anyOf/0", value))
                    if (typeof value !== "string") {
                        if (value === null) {
                            json += JSON_STR_EMPTY_STRING;
                        } else if (value instanceof Date) {
                            json += JSON_STR_QUOTE + value.toISOString() + JSON_STR_QUOTE;
                        } else if (value instanceof RegExp) {
                            json += serializer.asString(value.source);
                        } else {
                            json += serializer.asString(value.toString());
                        }
                    } else {
                        json += serializer.asString(value);
                    }
                else if (validator.validate("__fjs_root_3#/items/0/anyOf/1", value))
                    json += serializer.asNumber(value);
                else
                    throw new TypeError(
                        `The value of '#/items/0' does not match schema definition.`
                    );

                if (0 < arrayEnd) {
                    json += JSON_STR_COMMA;
                }
            } else {
                throw new Error(`Item at 0 does not match schema definition.`);
            }
        }
        value = obj[1];
        if (1 < arrayLength) {
            if (undefined) {
                if (validator.validate("__fjs_root_3#/items/1/anyOf/0", value))
                    if (typeof value !== "string") {
                        if (value === null) {
                            json += JSON_STR_EMPTY_STRING;
                        } else if (value instanceof Date) {
                            json += JSON_STR_QUOTE + value.toISOString() + JSON_STR_QUOTE;
                        } else if (value instanceof RegExp) {
                            json += serializer.asString(value.source);
                        } else {
                            json += serializer.asString(value.toString());
                        }
                    } else {
                        json += serializer.asString(value);
                    }
                else if (validator.validate("__fjs_root_3#/items/1/anyOf/1", value))
                    json += serializer.asNumber(value);
                else
                    throw new TypeError(
                        `The value of '#/items/1' does not match schema definition.`
                    );

                if (1 < arrayEnd) {
                    json += JSON_STR_COMMA;
                }
            } else {
                throw new Error(`Item at 1 does not match schema definition.`);
            }
        }

        return JSON_STR_BEGIN_ARRAY + json + JSON_STR_END_ARRAY;
    }
    const main = anonymous0;
    return main;
});

The schema serializer above can be reproduced by either:

setting break point at node_modules/fastify/lib/reply.js

function serialize (context, data, statusCode, contentType) {
  const fnSerialize = getSchemaSerializer(context, statusCode, contentType)
  if (fnSerialize) {
    return fnSerialize(data) // ===== step into this function ======
  }
  return JSON.stringify(data)
}

generate the schema with Typebox and serialize with fast-json-stringify

const FastJson = require('fast-json-stringify');
const { Type } = require('@sinclair/typebox');
const { TypeCompiler } = require('@sinclair/typebox/compiler');

const StringOrNum = Type.Union([Type.Number(), Type.String()]);
const UnionTuple = Type.Tuple([StringOrNum, StringOrNum]);

const schema = TypeCompiler.Compile(UnionTuple ).schema;
console.log(JSON.stringify(schema));

const serializer = FastJson(schema);
console.log(serializer([1, 2])); // ===== step into this function ======

why does fast-json-stringify fails at generating the correct code? What's the input JSON Schema that is causing this?

Tuple<Union> generates the schema below
{
  type: 'array',
  items: [
    { anyOf: [{ type: 'number' }, { type: 'string' }] },
    { anyOf: [{ type: 'number' }, { type: 'string' }] },
  ],
  additionalItems: false,
  minItems: 2,
  maxItems: 2,
}

Array<Union> generates this which works fine
{
  type: 'array',
  items: {
    anyOf: [{ type: 'number' }, { type: 'string' }],
  },
}

A PR with a fix for this would be really be nice. I guess items with multiple anyOf in it doesn't work.

repo: fastify/fastify-type-provider-typebox


text:
Feature: add language japanese

What is the feature?
Add Japanese version to homepage
Why is this feature needed?
for internationalization
Any additional comments?
No response
Confirmations

[X] I have read the Code of Conduct
[X] I have read the Issue Reporting Guidelines
[X] I have searched for similar issues in both open and closed tickets and cannot find a duplicate
[X] I know that any inappropriate content may cause the maintainer to take action base on the Code of Conduct
[ ] This feature request is open because to fix a bug/issue

Sure, but we will do this later, you can open a Pull Request if you want to do it yourself too.