text:
Expand deployment guide

Discuss deployments using some of the more popular web servers and cloud providers. This guide should only provide high-level guidance, and defer to the relevant project/vendor docs for details.
WSGI servers

uWSGI
Gunicorn
Twisted
httpd

Short discussion on reverse proxies, load balancers:

HAProxy
Nginx
httpd
lighttpd
Appliances (e.g., F5)
IaaS (e.g., ELB, Google Cloud LB)

Cloud providers:

AWS
Google
Microsoft
Digital Ocean
Heroku

See also:

http://bottlepy.org/docs/dev/deployment.html
http://werkzeug.pocoo.org/docs/0.11/deployment/
https://redmine.lighttpd.net/projects/lighttpd2/wiki/Howto_WSGI

@kgriffs can I take this on me?

@anekix go for it, thanks!

Hi @anekix, just following up, were you still interested in taking this one?

OK, I'm assuming this one is fair game for anyone else who would like to pick it up. :)

I have some examples of working with uWSGI & NGINX. What would be most helpful? An example configuration file and maybe a wsgi.py file for how to attach an app to an uWSGI master?

Related: https://github.com/falconry/falcon/issues/899

@nZac I think it would be great to have a page added to the doc with sections for difference deployment scenarious and a TOC at the top. uwsgi + nginx would be a great first addition to that page.

For Gunicorn, you can reference the official Deploying Gunicorn docs which I found very helpful recently.
I think it would be good to make a recommendation on which workers are recommended.  In my testing of a little hello world app, meinheld workers performed the best with Falcon.
           falcon
--------- ---------
eventlet     19.94k
gevent       19.95k
gunicorn     13.28k
meinheld     70.62k
uwsgi        errors

I would really like to see some recommendations on uWSGI settings too as I would constantly see socket errors when smashing uWSGI with load.
Tested on the following instance:
AMI: CentOS 7 (x86_64) - with Updates HVM
Type: t2.2xlarge
vCPUs: 8
Memory (GiB): 32
Python: 3.6.5 (from IUS YUM repository)
With the following test command:
wrk --duration 10s --threads 1 --connections 200 http://wsgi:8000

And the following Gunicorn command:
gunicorn \
  --workers 17 --worker-class=meinheld.gmeinheld.MeinheldWorker \
  --bind 0.0.0.0:8000 \
  hello_falcon:app```

And finally here's the Falcon code:

```python
import falcon

class HelloResource(object):
    def on_get(self, req, resp):
        resp.body = 'Hello World!'

app = falcon.API()
app.add_route('/', HelloResource())

To be honest, in all my testing thus far, I'm just not convinced of PyPy's performance once you add the WSGI server in, hence my issue #1309 so I'm keen to see your suggestions on deploying with PyPy too 😄
Keep up the great work!
Fotis

Hope it's OK if I post another little tip I've learned here.  If you are deploying with Gunicorn, it's important that you configure logging to write to Gunicorn's error logger or it won't log correctly during deployment.
To make this easy, I add the handler to the root logger like so:
# Configure the root handler log level and ensure that it is set to send logs
# to Gunicorn's error logs.
gunicorn_error_logger = logging.getLogger('gunicorn.error')
logging.root.handlers.extend(gunicorn_error_logger.handlers)
logging.root.setLevel(gunicorn_error_logger.level)

Perhaps the suggestion could then be that resources simply inherit this log level:
logger = logging.getLogger('app.endpoint')
logger.setLevel(logging.root.level)

Or of course, they can override it.
Hope this helps
Fotis

I find the gunicorn + meinheld deployment very easy to set-up
@see https://github.com/the-benchmarker/web-frameworks/blob/master/python/falcon/Dockerfile#L12

Hi @fgimian, did you managed to have PYPY to work with Falcon + Gunicorn?

authors: B1gG anekix fgimian kgriffs nZac waghanza

repo: falconry/falcon


text:
feat(testing): Add TestClient class and a pytest example

Add a contextual wrapper for the simulate_* methods and a pytest
example demonstrating its use.
Also deprecate "api" in favor of "app" in TestCase, since these utils
can work with any WSGI app (not just those that express HTTP APIs). This
terminology makes TestCase more consistent with the new simulate_*
functions and with TestClient.
Finally, deprecate "api_class" in TestCase and update the docstring
for the testing module to demonstrate a less hacky alternative for
controlling the api instance type. As a bonus, this avoids having to
try and alias api_class (a class variable).
Partially implements #16

Current coverage is 99.94% (diff: 96.15%)

Merging #862 into master will decrease coverage by 0.05%

@@           master       #862   diff @@
========================================
  Files          30         30
  Lines        1866       1873     +7
  Methods         0          0
  Messages        0          0
  Branches      305        305
========================================
+ Hits         1866       1872     +6
- Misses          0          1     +1
  Partials        0          0

Powered by Codecov. Last update 498b4b6...66a6898

:+1:

source: gharchive/pull-request

authors: codecov-io jmvrbanac kgriffs

license_type: permissive

license_source: github-api


text:
Some documentation pages easily get out of sync with Falco (could be auto generated)

/area documentation
What would you like to be added:
We are changing Falco fast enough that some pages become outdated very quickly, and we need the docs to keep up, especially in the reference section. These are the ones I found now:

https://falco.org/docs/reference/daemon/cli-arguments/ (perhaps it could be removed?)
https://falco.org/docs/reference/daemon/config-options/ (gets quickly out of date and is out of sync with falco.yaml, could be auto generated or removed?)
https://falco.org/docs/reference/rules/default-macros/
https://falco.org/docs/reference/rules/macros-override/
https://falco.org/docs/reference/rules/supported-events/ (this is already auto generated but the PR needs to be opened manually at each release, I wonder if we can do better)
https://falco.org/docs/reference/rules/supported-fields/ (this is already auto generated but the PR needs to be opened manually at each release, I wonder if we can do better)

Why is this needed:

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

/remove-lifecycle rotten

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

source: gharchive/issue

authors: LucaGuerra poiana

repo: falcosecurity/falco-website


text:
[Tracking] Opening multiple event sources in the same Falco instance

Motivation
The plugin system allows Falco to open new kinds of event sources that go beyond the historical syscall use case. Recently, this has been leveraged to port the k8s audit log event source to a plugin (see: https://github.com/falcosecurity/plugins/tree/master/plugins/k8saudit, and https://github.com/falcosecurity/falco/pull/1952). One of the core limitation that comes from the plugin system implementation of the libraries, is that a given Falco instance is capable of opening only one event source. In the example above, this implies that Falco instances are not able to ingest both syscalls and k8s audit logs together. This can instead be accomplished by deploying two distinct Falco instances, one for each event source.
Feature Requirements

(R1) A single Falco instance should become able to open more than one event source at once and in parallel
(R2) There should be feature parity and performance parity between having 2+ source active in parallel in a single Falco instances and having 2+ single-source Falco instances with the same event sources

Proposed Solution
Release Goals
To be defined. This is out of reach for Falco 0.32.1.
Terminology

Capture Mode: A configuration of sinsp inspectors that reads event from a trace file
Live Mode: A configuration of sinsp inspectors that reads event from one of the supported modes (kmod, ebpf, gvisor, plugin)

Design

(D1) The feature is implemented in Falco only, and mostly only affects the codebase of falcosecurity/falco. Both libsinsp and libscap will keep working in single-source mode
(D2) Falco manages multiple sinsp instances, one in each thread
(D3) Falco manages one or more instances of sinsp inspectors

If the # of inspectors is 1, everything runs in the main thread just like now
If the # of inspectors is 2+, each inspector runs in its own a separate thread (see (R1)). The whole event data path happens in parallel within each thread (event production, data enrichment, event-rule matching, and output formatting)

(D4) If in capture mode, Falco runs a only 1 inspector configured to read events from a trace file
(D5) If in live mode, Falco runs 1 inspector for each active event source

If an event source terminates due to EOF being reached, Falco waits for the other event sources to terminate too
If an event source terminates with an error, Falco forces the termination of all the other event sources

(D6) There is 1 instance of the Falco Rule Engine (just like now), and we leverage/enforce thread-safety guarantees to make sure it is safe and non-blocking for different threads to perform event-rule matching
(D7) There is 1 instance of the Falco Output Engine (just like now), and we leverage/enforce thread-safety guarantees to make sure it is safe for different threads to send alerts when an event-rule match is found

Non-blocking guarantees are less of a concern here, because the number of alerts is orders of magnitudes lower than the number of events

Technical Limitations of the Design

(L1) There cannot be 2+ event sources with the same name active at the same time

This would defeat the thread-safety guarantees of the Rule Engine, which are based on the notion of event source partitioning
Potential Workarounds (for the future, just in case):

Have more than one instances of the Rule Engine to handle the increased event source cardinality. For example, the second Rule Engine instance would cover all the second event source replicas, the third Rule Engine instance will handle the third replicas, and so on
We make the Rule Engine thread safe without the event source <-> thread 1-1 mapping assumptions. This is hardly achievable, because this would imply making the whole filtercheck system of libsinsp thread-safe too. Another naive solution would be to create one mutex for each event source to protect the access to the Rule Engine. In both scenarios, this would be hard to manage and performance would be sub-optimal
We have one Rule Engine for each source, which could become harder to manager. For example, rule files would need to be loaded by all the rule engines, which makes the initialization phase and hot-reloading slower too. However, this is something we can consider for the future.

(L2) Filterchecks cannot be shared across different event sources to guarantee thread-safety in the Rule Engine. The direct implication is that if an plugin with extractor capability is compatible with 2+ active event sources (e.g. json can extract from both aws_cloudtrail and k8s_audit), we need to create and initialize two different instances of the plugin (1 for each inspector)

Practically, this means that a given plugin instance will always extract fields coming from the same event source (a.k.a. subsequent calls to plugin_extract_fields will never receive events from two distinct event sources for the same initialized pluginstate)
This limitation can actually be turned as a by-design feature, because doing the contrary would violate (R2)
Potential Workarounds (for the future, just in case):

Make field extraction thread-safe (hardly doable, see points in (L1)

Technical Blockers
This is the list of things we mandatorily need to work on to see this initiative happen.

[ ] (B1) The rule engine <-> inspector source index mapping needs to be handle in different ways for capture mode and live mode

In capture mode, the rule engine source index is the same as the source index in the plugin manager of the single inspector used in capture mode (with the exception of the syscall source, which is by convention the last source index after all the plugin ones)
In live mode, each rule engine source index should be uniquely assigned to each inspector in live mode that runs in its own thread

[ ] (B2) Plugins can potentially be loaded multiple times in order to be registered to each live mode inspector

In live mode, a single plugin with field extraction capability can be registered to all the inspectors configured with an event  source they are compatible with

Note: This also applies to plugins with both field extraction and event sourcing capabilities. In this case, the plugin is register and used both its capabilities only with the inspector in which its event source is active, whereas it is just registered for its field extraction capability in all other event-source-compatible inspectors.

[ ] (B3)  The plugin API and the Plugin SDK Go should be revised to support multi-thread and concurrency assumptions (will likely be the only change needed outside of falcosecurity/falco)

Plugin API:

Most API symbols will need to support being called concurrently, with every call having distinct ss_plugin_t*s

Plugin SDK Go:

Tracking the discussion in another issue: https://github.com/falcosecurity/plugin-sdk-go/issues/62

[ ] (B4) Print-only Falco actions (e.g. list fields, list events, etc...) are dependent on the app state inspector. These needs to be stateless and can be implemented by allocating a sinsp inspector on-the-fly because they just access static information

https://github.com/falcosecurity/falco/pull/2097

[ ] (B5) The Falco StatsWriter (-s option) is not thread-safe

https://github.com/falcosecurity/falco/pull/2109

[ ] (B6) The Falco Rule Engine is does not provide any thread-safety guarantee

https://github.com/falcosecurity/falco/pull/2081 (non-blocker)
https://github.com/falcosecurity/falco/pull/2082

[ ] (B7) Signal-based actions (termination, restart, and output reopening) are not thread safe

https://github.com/falcosecurity/falco/pull/2091

[ ] (B8) The Falco Output framework is not entirely thread safe

https://github.com/falcosecurity/falco/pull/2080

[x] (B9) Libsinsp and libscap have a bunch of global and static variables that limit our freedom of having multiple inspectors running in parallel (g_infotables, g_logger, g_initializer, g_filterlist, g_decoderlist, s_cri_xxx, libsinsp::grpc_channel_registry::s_channels, s_callback, g_event_info, g_ppm_events, g_chisel_dirs, g_chisel_initializer, g_syscall_code_routing_table, g_syscall_table, g_syscall_info_table, g_json_error_log)

This may seem like a lot, but we should good to go as-is because most of these are read-only tables or objects. The ones that actually bundle some logic are either thread-safe (g_logger), or used only by inspectors running the syscall event source. Since due to (L1) we don't allow two inspectors running the syscall source at the same time, it should be safe to assume that no concurrent access will happen to the syscall-related globals.

[ ] (B10) The whole Falco application logic should be revised to support multiple inspectors, multiple filtercheck factories, and to distinguish the capture-mode and live-mode use cases as defined in the Design section. This will required all previous (BXX) points to be satisfied first

Nice to Have

[ ] (N1) Add a new -–enable-source=xxx option as a dual to -–disable-source=xxx. This design implies that the active event sources are chosen in an opt-out fashion: every loaded source gets activated, with the exception of the disabled ones. The -–enable-source option will make the UX better to define the only source users want to activate

https://github.com/falcosecurity/falco/pull/2085

[ ] (N2) Improve the regression testing framework falco_test.py to support selecting the active event source. Without it, all non-syscall tests will hang or fail, because the syscall event source is implicitly activated along with the testing-subject one and will cause Falco to not terminate (example: k8s audit tests)

https://github.com/falcosecurity/falco/pull/2085

[ ] (N3) Reduce threadiness of /healtz webserver (based on cpp-httplib). The webserver library documents that the default threadiness is 8 or std::thread::hardware_concurrency(). This is OK, but since we go on a multi-threaded model we should consider limiting the number of threads spawned by Falco. In my 8-core setup, I Falco spawned 30 threads (only few were active, luckily) with a simple test with syscalls and 2 Go plugins loaded.

https://github.com/falcosecurity/falco/pull/2090

Linked Discussions

https://kubernetes.slack.com/archives/CMWH3EH32/p1655762074442099
https://kubernetes.slack.com/archives/CMWH3EH32/p1655209558876649
https://kubernetes.slack.com/archives/CMWH3EH32/p1647969834426869
https://kubernetes.slack.com/archives/CMWH3EH32/p1646148516502339?thread_ts=1646061690.259599&cid=CMWH3EH32
https://kubernetes.slack.com/archives/CMWH3EH32/p1645610199932789?thread_ts=1645396079.322259&cid=CMWH3EH32
https://kubernetes.slack.com/archives/CMWH3EH32/p1645038583559959?thread_ts=1645034669.067289&cid=CMWH3EH32
https://github.com/falcosecurity/falco/issues/2110

I think this is something we should aim for over the next few months. So, I propose to turn this issue into a tracking issue where I'll share the though process of my proposed solution and all the steps and PRs involved. I expect this to be the place in which we can have discussions about this solution, and perhaps converge to something we all agree on.
/milestone 0.33.0

I'm gonna share the thought process that led me to the proposed solution documented in the issue description. @leogr has been involved in the early discussion of this topic.
Since the end goal is to run multiple sources at the same time, many execution models have been hypothesized, but each of them has been discarded in favor of the proposed solution. I'm gonna proceed and document briefly their design and cost/benefits tradeoffs.

(H1) Single-threaded model, with multiple event sources opened by libscap

This would be simple to implement, but would require lots of refactorings of the libraries
This would not work in the general case, because performance would be tremendously degraded depending on the event consumption policy of libscap (e.g. round-robin)
Each event source would be blocked by the other ones. Since there are no buffering or multi-threading, every event production request is sequential. This means that "slow" event sources would become the bottleneck for all the other ones. In the case of syscalls, this would be totally unacceptable in terms of event drop metrics

(H2) Multi-threaded model, with parallelization at the libscap level: libscap would become able to open multiple event sources in parallel and consume all of them from a single thread

Not too hard to implement
"Slow" event sources would not impact the others anymore
Producing events in parallel would require a queue-like synchronization system, similar to our syscall ringbuffer, which would also require backpressure control support and memory copies. Both of these would degrade performance to a non-acceptable level

(H3) Multi-threaded model, with parallelization at the libsinsp level: libsinsp manages multiple scap instances in parallel

Costs and benefits are totally analoguous to (H2). In this case, libsinsp would be come the sequential bottleneck

Reasonably, going up in the stack the next synchronization point would be Falco itself, which is exactly the solution we proposed. This is the only design we came up with that ensure total and non-blocking parallelization of each event source processing routine. The only sequential bottleneck becomes the Falco Output Engine, which is natively designed to be thread-safe(~ish), and has the benefits that the number of alerts is orders of magnitude lower than the number of events produced by each event source (syscalls in particular), which ensure acceptable performance and negligible synchronization overhead.

@araujof thanks for reviewing, those are all good points!
For D6, we can achieve good thread safety guarantees from the Rule Engine with the assumption that every thread gets assigned different event source. This stands before the Rule Engine is entirely partitioned by event source. See this for more details: https://github.com/falcosecurity/falco/pull/2082. Assuming that the Rule Engine level can provide wait-free concurrent access, I'd say fairness should be guaranteed.
Then, once alerts are triggered they go through the Output Engine, where fairness and ordering are a big topic instead. Luckily, thanks to @leogr we already have a concurrent ordered queue there. We only have one worker on the other side of the queue though, and we might consider using a thread pool maybe in the future.
What do you think? Does this address your concerns?

Hi @jasondellaluce,

What do you think? Does this address your concerns?

Sounds good to me! Thanks for the pointer to #2082.

Thanks for the very thorough write up! I completely agree that this is the best/least-bad approach to take. Here are some more specific comments:
(D1): you're right that only falco will be opening multiple inspectors. But (B9) is going to result in a lot of changes to make opening multiple inspectors possible.
(D4): outside of test cases/ad-hoc testing, I don't think anyone would run falco on a trace file, right? Might want to treat this as a special case and not try too hard to support multiple inspectors.
(B3): I don't recall the implementation, but why can't you partition around handles or inspectors to avoid making the async readers concurrent?
(B6): Any concurrency improvements will be limited to partitioning by event source, right?
(B8): Instead of removing the token bucket, you could move it to the other side of the outputs queue, right?
(B9): You're right that these might be safe because they are read-only, but I think this would be a great time to move them into the inspector, simply as a cleanup step. I think there are some other potential globals like the dns manager singleton that need to be cleaned up as well.

Hey Mark, responding point by point:

I tried to make deep research on this, and my opinion is that (B9) is not a blocker for (D1) with the assumption that only one inspector opens the syscall event source. With this assumption, all the static and globals in libsinsp are either read-only (e.g. all the tables), used only for syscalls (e.g. container engine), or thread-safe (e.g. g_logger). So as long as we have (L1) we should be good to go for this I think
I agree, capture files should be the core focus of this but we still need to support the feature. This case is quite easy to manage as it only has to use one inspector, it's just handled differently than the live mode case
The issue with async extractor is that we have only 1 async worker running for extraction. So far, we only had one ss_plugin_t* active per time for each plugin, so this was a single-consumer-single-producer model. Now that a given plugin can be initialize 2+ times at the same time (e.g. json extracts fields from 2 different event sources), this can in general become a multiple-consumer-single-producer model, which the code is not ready to handle. The approach you're proposing here seems like the Hard proposed solution that I listed (1 worker for each plugin instance), which is OK but requires lots of testing to make sure all those workers don't end up consuming tons of CPU time. I think option 1 or 2 of the proposed solutions might be a good start for the first release of this feature
Right, partitioning the engine by event source like we already do should be enough. The only exception is the stats_manager internal class, which would be shared across all the sources. But that's an easy fix, because we can turn each counter into an atomic one with relaxed memory order guarantee. Moreover, this should not impact performance because it gets into play only when a rule is triggered. See #2082 for more context.
Yes, I could move the token bucket on the other side of the queue, but this would mean sharing it across all the active event sources, which opens the door to many security concerns. Let's say that we have a very noisy event source+ruleset, we risk that alerts coming from other event sources get discarded due to the shared rate limiter. I strongly advocate that event sources should not interfere with each other as much as possible, also because this would assume a different behavior if the same event sources are run in distinct Falco instances (and we want to achieve feature parity). Besides, the rate limiter has been indicated many times as risky for the scope of security, so I advocate that dropping it is the cleanest solution here.
This also connects with your first point. I totally agree with this, removing static and global variables from libsinsp is a cleanup we should really look up for. However, I expect it to be a good cleanup effort because many of them are very eradicated in many parts of the code (e.g. g_filterchecklist). However, the way this feature is designed makes its development orthogonal to this kind of cleanup. In this way, both efforts can proceed without blocking each other.

Thanks for reviewing @terylt! This is actually a very valid suggestion, and It's something we should consider. The clear benefit of this is that multiple event sources with the same name could be loaded (with the exception of syscall, that carries other technical constraints), and that falco_engine would not need to be thread-safe anymore.
The downside of this is that loads would need to be loaded multiple times (slower initialization time). Cloning the engine would not work, because each engine would be configured with a given event source so they would all be different and rely on distinct sets of available rule fields. Also, it would make hot-reload features like the one described in https://github.com/falcosecurity/falco/issues/1692#issuecomment-1110951413 harder to implement.
At the same time, this is the one change we can consider making even later into the process. So since we don't have data races in our current Engine implementation, my call would be to have a first feature release that uses only one engine, and eventually switch to multiple ones if we find limitations while experimenting. What do you think?

@jasondellaluce What takes the longest for initialization of the engine?  Is it compiling the rules into a datastructure?  If so, is that data structure read-only?  I.e., could it be shared across multiple engines?   That way you'd only have to compile it once, and share it.  If it's not being modified by any of the engines, you could get away with one.  If we have a list of major initialization costs, we could figure out ways to reduce each one.
In terms of multi engine vs. single engine.  I don't have a strong opinion and am happy with whatever route you want to take.   Just sharing from my past experience on this topic.  I find pushing all events into a single pipe can be harder to get right coding-wise and can introduce overheads in context-switching and lock contention, so I tend to gravitate to avoid locking and contention issues when I can.   I'm happy to help or support in anyway I can during the building of this feature, whichever approach you decide.

@terylt the initialization cost of the rule engine is related to loading the rule files themselves. If we decide to have one engine for each event source, each engine should load all the configured rule files, because the way rule files are loaded is influenced by the event sources known by the rule engine. For example, the engine throws errors if a rule uses a field that is not defined for the rule's event source, and skips rules of unknown event source. There would be nothing to share between engines, if not lists and rules abstract definitions. Plus, currently rule files reading and compilation into an evaluable data structure currently happens in one step, so this approach would require relevant refactoring at that level.
The reason why I didn't yet do deeper research on this route is that having a 1 shared rule engine still does not require any locking mechanism in our case. The rule engine is currently totally partitioned by event source, so as long as we assign 1 event source to 1 thread, everything will work as-is. The only synchronization point is the rule stats manager inside the rule engine, which can just be turned into an atomic counter.

@dwindsor thanks for reviewing!

Also late to the party, but we have a use case for wanting support for this at the libs level (using the libs as the basis for a collector/sensor, but none of the other bits). Might not be a common use case, but figured I'd mention it 😀.

No worries, we're still in the process of designing and developing this. As for the analysis of https://github.com/falcosecurity/falco/issues/2074#issuecomment-1162922363, I couldn't find an optimal way to support event source parallelism in neither the libscap nor the libsinsp levels.
I think your point is very valid, supporting this in libs could be valuable for all the libs stakeholder. Do you have something specific in mind? Given the analysis linked above, here's my proposal. We could maintain the assumption that each sinsp inspector is single-source and single-thread, and then created another library inside libs that implements the inspector parallelization logic. Such a library would sit inside something like userspace/libsinp-multi-source, and would become optionally usable for consumers that wish to run more than one inspector, with different event sources, and eventually with multi-threading. This would make the Falco code easier to write, would have no effect on current libsinsp/libscap implementations, and would also enable libs consumer to benefit from this new feature. Let me know if this would work for you, so that I can proceed with a POC.

Looking at the discussion around (B9) my vote would be to go with the @mstemm's suggestion and try to find a way to cleanly handle the globals, and (perhaps?) be able to go the route of adding libs-level support?

I agree with both you and @mstemm. My opinion here is that these (Falco multi-evt source and libs globals cleanup) should follow two separate workstreams and efforts. The former could be properly implemented even without the latter, so I'd propose to work on both, even in parallel if there is enough developer capacity. My fear is that removing sinsp's globals will be a bit tedious.

FYI, I moved the discussion of point (B3) about the Plugin SDK Go into another dedicated issue. There are some interesting challenges involved, and I felt like it was worth documenting the thought process so that everyone could jump in and share some feedback.
👉🏼  https://github.com/falcosecurity/plugin-sdk-go/issues/62

Just my 2c here!
I agree: we can provide an abstraction library on top of libsinsp; at the very same time, i don't really think it matters: since "opening a sinsp inspector" won't require any synchronization, clients of the libs can just open multiple inspectors binding each of them to a new thread.
Therefore, i can't really see the practicality of yet another library just to abstract multi-libsinsp support. In any case, supporting something like that should not be a big deal in the end.

My opinion here is that these (Falco multi-evt source and libs globals cleanup) should follow two separate workstreams and efforts. The former could be properly implemented even without the latter, so I'd propose to work on both, even in parallel if there is enough developer capacity. My fear is that removing sinsp's globals will be a bit tedious.

Agree with this also! We would be thrilled to see contributions in this sense too :)
Btw this is a huge effort and will be the feature of Falco 0.33 IMHO, and this discussion seems to confirm that! So glad @jasondellaluce started this thread :)

Do you have something specific in mind?

@jasondellaluce thanks for the detailed analysis 🙏. Thinking about this more, I'm not sure our use case should be addressed by adding libs-level support for ingesting plugin data.
We currently are collecting syscall telemetry using libsinsp, serializing the telemetry and dispatching it to an analysis service, which then generates alerts, etc based upon state collected.
The use case I was thinking of was to write a plugin e.g. a uprobe that attaches to libssl and returns connection-related data, and having that plugin's data routed through libsinsp so we can receive that telemetry without installing full Falco.
But, I feel that libsinsp might not be the appropriate place for handling plugin logic. sinsp, to me, is for low-level system inspection (syscalls, etc) - highly granular data that comes in at a very high rate. SSL connection data doesn't feel like it belongs in a system inspect library.

We could maintain the assumption that each sinsp inspector is single-source and single-thread, and then created another library inside libs that implements the inspector parallelization logic.

Yeah, we'd need another level of indirection here, and also account for the hugely different speeds of events coming in... I agree with @FedeDP that it wouldn't be worth it.
We're already able to ingest other forms of telemetry alongside libsinsp syscall telemetry, so no big deal! 😀

The use case I was thinking of was to write a plugin e.g. a uprobe that attaches to libssl and returns connection-related data, and having that plugin's data routed through libsinsp so we can receive that telemetry without installing full Falco.

@dwindsor I think your use case would be well suited to be implemented as a plugin. Falco plugins are actually "libs" plugins: the plugin framework is implemented at the libscap and libsinsp level. In fact, the goal of the plugin system was to allow collecting events and extracting fields from them from any kind of event source, just like we traditionally did for syscalls.

Yeah, we'd need another level of indirection here, and also account for the hugely different speeds of events coming in... I agree with @FedeDP that it wouldn't be worth it.

Agreed on avoiding another level of indirection. The "different speeds of events coming in" sort of confirms the decision of having single-source-single-thread sinsp inspectors. With this assumption, the different event rates is not relevant as you can have multiple inspectors running in parallel each with dedicated thread and event source, and then collect events from each of them whenever available. In this model, event sources do not influence each other and the only "sequential bottleneck" is the final event collection point, which will likely happen in a shared place. This is what I plan to do with Falco, with the extra step of evaluating event rules in parallel too and then collecting only matching rule alerts, which have a fairly lower rate.
In this perspective, I think the use case you described could be achieved by:

Developing a plugin in either C/C++ or Go to collect SSL connection-related data and defining fields to be extracted from that kind of event
In your application, spawning two distinct sinsp inspectors in two threads, one configured with your new plugin and one for syscall collection
Collecting and serializing the telemetry from both inspectors and dispatching it to your analysis service

In this way, you'd collect both types of events in the same way. I think this could be valuable for the community too, I would be happy to help if you think this route makes sense to you.

This are moving forward! Just opened https://github.com/falcosecurity/falco/pull/2182, which should be the final piece of glue code that uses all the preliminary work to implement the multi-source feature.

https://github.com/falcosecurity/falco/pull/2182 just got merged! 🥳
Will keep this open until the testing phase is over, just in case some extra changes are needed!

authors: FedeDP araujof dwindsor jasondellaluce mstemm terylt

repo: falcosecurity/falco


text:
cleanup: fix several warnings from a Clang build

What type of PR is this?
/kind cleanup
Any specific area of the project related to this PR?
What this PR does / why we need it:
A Clang build revealed a number of warnings whose fix should increase the quality of the code.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
NONE

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: federico-sysdig
Once this PR has been reviewed and has the lgtm label, please assign lucaguerra for approval. For more information see the Kubernetes Code Review Process.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

The fix is only a cleanup of warnings that introduces no interface of functionality change.
/unhold

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: federico-sysdig, LucaGuerra
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [LucaGuerra]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: dc570829f3e00c0e4de1159179b498d7bb10e1b2

@LucaGuerra: The provided milestone is not valid for this repository. Milestones in this repository: [0.37.0, 0.38.0, TBD]
Use /milestone clear to clear the milestone.

In response to this:

/milestone 0.14.0

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

/milestone 0.37.0

authors: LucaGuerra federico-sysdig poiana


text:
fix(userspace/libsinsp): solve fdtables 'type' field returning random data

What type of PR is this?
/kind bug
/kind feature
Any specific area of the project related to this PR?
/area libsinsp
Does this PR require a change in the driver versions?
What this PR does / why we need it:
I discovered that the type field exposed by the fd tables through the state & plugin API is returning wrong/unrelated data since it's been available. Unfortunately, IMO the safest fix is to turn its type into uint8, which is essentially a breaking change for whoever was accessing it (the type change will cause errors to be returned). However, considering the information returned has never been valid I think this is something we can afford, specially considering that the latest libs version has been released just few weeks ago.
I also noted that we're not exporting the fd field, which I just added being very trivial.
Which issue(s) this PR fixes:
Special notes for your reviewer:
/milestone 0.18.0
Does this PR introduce a user-facing change?:
fix(userspace/libsinsp): solve fdtables 'type' field returning random data

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: jasondellaluce
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: b1e549cabc346af3d5ca9b3590a3211b37115c6c

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, jasondellaluce
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, incertum, jasondellaluce
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,incertum,jasondellaluce]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

authors: jasondellaluce poiana

repo: falcosecurity/libs


text:
build: upgrade deps and cmakefiles improvements

What type of PR is this?
/kind feature
Any specific area of the project related to this PR?
/area build
What this PR does / why we need it:
This PR upgrades c-ares, protobuf and gRPC versions. It also changes the way we build gRPC when deps are bundled. The building of protobuf and c-ares is not delegated anymore to gRPC.
This PR also includes other changes (some required by the gRPC upgrade, some are just small improvements):

switching from gRPC _usecure libs to default ones
openSSL configured with no-shared
c-ares configured with --disable-shared

Finally, other changes are included to easier the libraries integration into the Falco build system, see also https://github.com/falcosecurity/falco/pull/1552
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Unfortunately, this PR introduces a workaround, see https://github.com/falcosecurity/libs/pull/30/files#diff-80c09a7dee6a7fbc4e1648084e71598c7e28978e00ff50ab9381e7b8e20ec8b8R49
Since this PR took longer than expected and is blocking https://github.com/falcosecurity/falco/pull/1552 I'd prefer to fix that later.
Does this PR introduce a user-facing change?:

build: switching from gRPC `_usecure` libs to default ones
build: upgrade gRPC to 1.32.0 when bundled
build: upgrade protobuf to 3.13.0 when bundled
build: upgrade c-ares to 1.15.0 when bundled
build: openSSL configured with `no-shared` when bundled
build: curl configure without brotli

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To complete the pull request process, please ask for approval from leogr after the PR has been reviewed.
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Update: I'm re-working this PR again

@leogr: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

build-libs-bundled-deps
2e009d81f2c6cd07f1c14e89619b38d45a26c665
link
/test build-libs-bundled-deps

Full PR test history. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@leogr: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

build-libs-bundled-deps
2e009d81f2c6cd07f1c14e89619b38d45a26c665
link
/test build-libs-bundled-deps

build-libs
2e009d81f2c6cd07f1c14e89619b38d45a26c665
link
/test build-libs

Full PR test history. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@leogr: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

build-libs-bundled-deps
2e009d81f2c6cd07f1c14e89619b38d45a26c665
link
/test build-libs-bundled-deps

build-libs
2e009d81f2c6cd07f1c14e89619b38d45a26c665
link
/test build-libs

build-libs-with-chisels
2e009d81f2c6cd07f1c14e89619b38d45a26c665
link
/test build-libs-with-chisels

Full PR test history. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

This PR is ready for review
/cc @fntlnz @gnosek @leodido

/approve

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leogr
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

authors: leogr poiana


text:
fix(kmod): support clone child exit events on s390x

What type of PR is this?
/kind bug
/kind feature
Any specific area of the project related to this PR?
/area driver-kmod
/area driver-bpf
Does this PR require a change in the driver versions?
What this PR does / why we need it:
This PR introduces 2 main changes:

enable support for BPF raw_tracepoint on s390x architectures. BPF has some problems with architectures like s390x as also described in this issue https://github.com/falcosecurity/libs/issues/497.  With this fix the BPF probe can works also on kernel versions >= 4.17, but the problem is that there are many instrumentation drops due to the use of bpf_probe_read() and bpf_probe_read_str() that don't work at all on architectures like s390x as you can read here. In the end I would say that we cannot use bpf on these architectures, at least for the moment

enable the CAPTURE_SCHED_PROC_FORK logic in order to catch the clone_child exit events on s390x architectures. Like in ARM64, the clone child exit event is not traced by the syscalls:sys_exit tracepoint, while the excve exit event, in this case, is traced! So to reuse the same logic introduced in ARM64 we have just to enable it in this way:
#if defined(CONFIG_ARM64) || defined(CONFIG_S390)
    #define CAPTURE_SCHED_PROC_FORK
#endif

For more info about this problem of the clone_child exit event take a look at the ARM64 fix.

Which issue(s) this PR fixes:
Special notes for your reviewer:
Does this PR introduce a user-facing change?:
fix(kmod): support `clone child` exit events on `s390x`

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

This is truly a 💣 !
I'd also update the README about the supported architectures and the minimum kmod version + an eBPF note!
About the supported architectures: IMHO we can have libs repo that supports more architectures than Falco officially supports; it just means that Falco would work on those but has no officially provided packages + drivers builds.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 78a3ec8390bcac2e75361854c199e717bc7a49ae

LGTM label has been added.  Git tree hash: cbaf552d708b61f34a5ada5882260d106963bab3

Also, I really appreciate you taking the time to link the relevant kernel code on comments, make understanding and tracking the changes much easier!

New changes are detected. LGTM label has been removed.

LGTM label has been added.  Git tree hash: d0d58e0af48d461ae24f8f375ca1fb46d3090bf8

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: Andreagit97, FedeDP, Molter73
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [Andreagit97,FedeDP,Molter73]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


text:
pdig compilation is failing

Describe the bug
While compiling pdig on Alpine env it's failing with error
make[2]: *** [CMakeFiles/pdig.dir/build.make:230: CMakeFiles/pdig.dir/source/libs/driver/ppm_fillers.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/pdig.dir/all] Error 2
make: *** [Makefile:91: all] Error 2
How to reproduce it
apk add --update
git
g++
gcc
cmake
make
libtool
elfutils-dev
libelf-static
linux-headers
git clone https://github.com/falcosecurity/pdig
git clone https://github.com/falcosecurity/libs
cd pdig && mkdir build && cd build
cmake -DMUSL_OPTIMIZED_BUILD=True ..
make
Expected behaviour
You can now find the pdig binary in the source directory you created under pdig/build/pdig
Environment
OS:

NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.12.1
PRETTY_NAME="Alpine Linux v3.12"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"
Installation method:
From source.

Additional context
bash-5.1# cmake -DMUSL_OPTIMIZED_BUILD=True ..
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
Compatibility with CMake < 2.8.12 will be removed from a future version of
CMake.
Update the VERSION argument  value or use a ... suffix to tell
CMake that the project does not need compatibility with older versions.
-- The C compiler identification is GNU 10.3.1
-- The CXX compiler identification is GNU 10.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /source/pdig/build
bash-5.1# make
[  6%] Building CXX object CMakeFiles/pdig.dir/pdig.cc.o
[ 12%] Building C object CMakeFiles/pdig.dir/pdig_compat.c.o
In file included from /source/pdig/pdig_compat.c:12:
/usr/include/sys/signal.h:1:2: warning: #warning redirecting incorrect #include <sys/signal.h> to <signal.h> [-Wcpp]
1 | #warning redirecting incorrect #include <sys/signal.h> to <signal.h>
|  ^~~~~~~
In file included from /source/pdig/pdig_compat.c:23:
/source/pdig/udig_capture.h:25: warning: "F_CANCELLK" redefined
25 | #define F_CANCELLK (F_LINUX_SPECIFIC_BASE + 5)
|
In file included from /source/pdig/pdig_compat.c:14:
/usr/include/fcntl.h:127: note: this is the location of the previous definition
127 | #define F_CANCELLK 1029
|
/source/pdig/pdig_compat.c: In function 'record_event':
/source/pdig/pdig_compat.c:331:15: warning: pointer targets in assignment from 'uint8_t *' {aka 'unsigned char *'} to 'char *' differ in signedness [-Wpointer-sign]
331 |   args.buffer = g_ring + head + sizeof(struct ppm_evt_hdr);
|               ^
/source/pdig/pdig_compat.c:255:11: warning: variable 'usedspace' set but not used [-Wunused-but-set-variable]
255 |  uint32_t usedspace;
|           ^~~~~~~~~
[ 18%] Building C object CMakeFiles/pdig.dir/pdig_ptrace.c.o
[ 25%] Building CXX object CMakeFiles/pdig.dir/pdig_ptrace_amd64.cc.o
[ 31%] Building CXX object CMakeFiles/pdig.dir/pdig_seccomp.cc.o
[ 37%] Building CXX object CMakeFiles/pdig.dir/proc_tree.cc.o
In file included from /source/pdig/proc_tree.cc:10:
/usr/include/sys/signal.h:1:2: warning: #warning redirecting incorrect #include <sys/signal.h> to <signal.h> [-Wcpp]
1 | #warning redirecting incorrect #include <sys/signal.h> to <signal.h>
|  ^~~~~~~
[ 43%] Building C object CMakeFiles/pdig.dir/udig_procs.c.o
In file included from /source/pdig/udig_procs.c:28:
/source/pdig/udig_capture.h:25: warning: "F_CANCELLK" redefined
25 | #define F_CANCELLK (F_LINUX_SPECIFIC_BASE + 5)
|
In file included from /source/pdig/udig_procs.c:14:
/usr/include/fcntl.h:127: note: this is the location of the previous definition
127 | #define F_CANCELLK 1029
|
/source/pdig/udig_procs.c: In function 'udig_proc_startupdate':
/source/pdig/udig_procs.c:496:40: warning: '%s' directive output may be truncated writing up to 255 bytes into a region of size 252 [-Wformat-truncation=]
496 |  snprintf(filename, sizeof(filename), "%sexe", dir_name);
|                                        ^~      ~~~~~~~~
/source/pdig/udig_procs.c:496:2: note: 'snprintf' output between 4 and 259 bytes into a destination of size 252
496 |  snprintf(filename, sizeof(filename), "%sexe", dir_name);
|  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/source/pdig/udig_procs.c:529:40: warning: '%s' directive output may be truncated writing up to 255 bytes into a region of size 252 [-Wformat-truncation=]
529 |  snprintf(filename, sizeof(filename), "%sstatus", dir_name);
|                                        ^~         ~~~~~~~~
/source/pdig/udig_procs.c:529:2: note: 'snprintf' output between 7 and 262 bytes into a destination of size 252
529 |  snprintf(filename, sizeof(filename), "%sstatus", dir_name);
|  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/source/pdig/udig_procs.c:556:40: warning: '%s' directive output may be truncated writing up to 255 bytes into a region of size 252 [-Wformat-truncation=]
556 |  snprintf(filename, sizeof(filename), "%scmdline", dir_name);
|                                        ^~          ~~~~~~~~
/source/pdig/udig_procs.c:556:2: note: 'snprintf' output between 8 and 263 bytes into a destination of size 252
556 |  snprintf(filename, sizeof(filename), "%scmdline", dir_name);
|  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/source/pdig/udig_procs.c:579:46: warning: '%s' directive output may be truncated writing up to 4095 bytes into a region of size 1024 [-Wformat-truncation=]
579 |    snprintf(tinfo->exe, SCAP_MAX_PATH_SIZE, "%s", line);
|                                              ^~   ~~~~
/source/pdig/udig_procs.c:579:4: note: 'snprintf' output between 1 and 4096 bytes into a destination of size 1024
579 |    snprintf(tinfo->exe, SCAP_MAX_PATH_SIZE, "%s", line);
|    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/source/pdig/udig_procs.c:603:40: warning: '%s' directive output may be truncated writing up to 255 bytes into a region of size 252 [-Wformat-truncation=]
603 |  snprintf(filename, sizeof(filename), "%senviron", dir_name);
|                                        ^~          ~~~~~~~~
/source/pdig/udig_procs.c:603:2: note: 'snprintf' output between 8 and 263 bytes into a destination of size 252
603 |  snprintf(filename, sizeof(filename), "%senviron", dir_name);
|  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[ 50%] Building C object CMakeFiles/pdig.dir/source/libs/driver/dynamic_params_table.c.o
[ 56%] Building C object CMakeFiles/pdig.dir/source/libs/driver/event_table.c.o
[ 62%] Building C object CMakeFiles/pdig.dir/source/libs/driver/flags_table.c.o
[ 68%] Building C object CMakeFiles/pdig.dir/source/libs/driver/ppm_events.c.o
In file included from /source/libs/driver/ppm_events.c:77:
/source/pdig/udig_capture.h:25: warning: "F_CANCELLK" redefined
25 | #define F_CANCELLK (F_LINUX_SPECIFIC_BASE + 5)
|
In file included from /source/libs/driver/ppm_events.c:50:
/usr/include/fcntl.h:127: note: this is the location of the previous definition
127 | #define F_CANCELLK 1029
|
/source/libs/driver/ppm_events.c: In function 'parse_readv_writev_bufs':
/source/libs/driver/ppm_events.c:1366:15: warning: 'val' may be used uninitialized in this function [-Wmaybe-uninitialized]
1366 |    args->fd = (int)val;
|               ^~~~~~~~
[ 75%] Building C object CMakeFiles/pdig.dir/source/libs/driver/ppm_fillers.c.o
In file included from /source/libs/driver/ppm_fillers.c:78:
/source/pdig/udig_capture.h:25: warning: "F_CANCELLK" redefined
25 | #define F_CANCELLK (F_LINUX_SPECIFIC_BASE + 5)
|
In file included from /source/libs/driver/ppm_fillers.c:54:
/usr/include/fcntl.h:127: note: this is the location of the previous definition
127 | #define F_CANCELLK 1029
|
/source/libs/driver/ppm_fillers.c: In function 'f_sys_socketpair_x':
/source/libs/driver/ppm_fillers.c:1595:15: warning: unused variable 'speer' [-Wunused-variable]
1595 |  struct sock *speer;
|               ^~~~~
/source/libs/driver/ppm_fillers.c:1594:20: warning: unused variable 'us' [-Wunused-variable]
1594 |  struct unix_sock *us;
|                    ^~
/source/libs/driver/ppm_fillers.c:1593:17: warning: unused variable 'sock' [-Wunused-variable]
1593 |  struct socket *sock;
|                 ^~~~
/source/libs/driver/ppm_fillers.c:1592:6: warning: unused variable 'err' [-Wunused-variable]
1592 |  int err;
|      ^~~
/source/libs/driver/ppm_fillers.c:1591:6: warning: unused variable 'fds' [-Wunused-variable]
1591 |  int fds[2];
|      ^~~
/source/libs/driver/ppm_fillers.c:1590:16: warning: unused variable 'val' [-Wunused-variable]
1590 |  unsigned long val;
|                ^~~
/source/libs/driver/ppm_fillers.c: In function 'f_sys_accept_x':
/source/libs/driver/ppm_fillers.c:1973:17: warning: unused variable 'sock' [-Wunused-variable]
1973 |  struct socket *sock;
|                 ^~~~
/source/libs/driver/ppm_fillers.c:1972:6: warning: unused variable 'err' [-Wunused-variable]
1972 |  int err = 0;
|      ^~~
/source/libs/driver/ppm_fillers.c: In function 'f_sys_pipe_x':
/source/libs/driver/ppm_fillers.c:2814:15: warning: unused variable 'file' [-Wunused-variable]
2814 |  struct file *file;
|               ^~~~
/source/libs/driver/ppm_fillers.c: In function 'f_sys_openat2_e':
/source/libs/driver/ppm_fillers.c:4616:18: error: storage size of 'how' isn't known
4616 |  struct open_how how;
|                  ^~~
/source/libs/driver/ppm_fillers.c:4650:53: error: invalid application of 'sizeof' to incomplete type 'struct open_how'
4650 |  res = ppm_copy_from_user(&how, (void *)val, sizeof(struct open_how));
|                                                     ^~~~~~
/source/libs/driver/ppm_fillers.c:4616:18: warning: unused variable 'how' [-Wunused-variable]
4616 |  struct open_how how;
|                  ^~~
/source/libs/driver/ppm_fillers.c: In function 'f_sys_openat2_x':
/source/libs/driver/ppm_fillers.c:4698:18: error: storage size of 'how' isn't known
4698 |  struct open_how how;
|                  ^~~
/source/libs/driver/ppm_fillers.c:4732:53: error: invalid application of 'sizeof' to incomplete type 'struct open_how'
4732 |  res = ppm_copy_from_user(&how, (void *)val, sizeof(struct open_how));
|                                                     ^~~~~~
/source/libs/driver/ppm_fillers.c:4698:18: warning: unused variable 'how' [-Wunused-variable]
4698 |  struct open_how how;
|                  ^~~
make[2]: *** [CMakeFiles/pdig.dir/build.make:230: CMakeFiles/pdig.dir/source/libs/driver/ppm_fillers.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:83: CMakeFiles/pdig.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

authors: poiana tmukharj

repo: falcosecurity/pdig


text:
Plugins not compiling with minimum defined go version in go.mod

Describe the bug

The minimum required go version in many plugin go.mod files is 1.15. If you actually try to compile the source with go 1.15 you get a compile error:
Error: /home/runner/go/pkg/mod/github.com/alecthomas/jsonschema@v0.0.0-20220216202328-9eeeec9d044b/comment_extractor.go:5:2: package io/fs is not in GOROOT

This io/fs package requires at least go 1.16. The CI workflows build with go 1.18 atm.
How to reproduce it

Try to build with the minimum required version 1.15 defined in go.mod, e.g. the github plugin.
Expected behaviour

Successful build
Screenshots

Environment

Falco version:

System info:

Cloud provider or hardware configuration:
OS:

Kernel:

Installation method:

Additional context

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle stale

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close.
Provide feedback via https://github.com/falcosecurity/community.
/lifecycle rotten

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://github.com/falcosecurity/community.
/close

@poiana: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.
Provide feedback via https://github.com/falcosecurity/community.
/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

authors: poiana sboschman

repo: falcosecurity/plugins


text:
Use an initial version of 0.1.0 for plugins/plugin api

Instead of starting with a bold 1.0.0 for the initial plugins api
version, we're going to start with a more reasonable 0.1.0. While
we're at it, let's use an initial version of 0.1.0 for all plugin
versions.
After this, the plugin versions will progress independently from each
other and from the api version.
Signed-off-by: Mark Stemm mark.stemm@gmail.com

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area source-plugins
/area extractor-plugins

/area build

/area documentation

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

NONE

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: mstemm
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [mstemm]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: leogr, mstemm
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [leogr,mstemm]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

LGTM label has been added.  Git tree hash: 683de5b7c835020b8bb474fade57dc32692fcffb

authors: mstemm poiana


text:
level editor.

all in the title.
for the basic proto of the editor:
push b in lvl select to access it.   dpad to move cursor.   select
to change block   a to place   b to remove   start to access the
editor pause menu   start again in pause menu to play current level
select in pause menu to move cursor to a button   b to go back to
editor   a in pause menu to choose your starting gamemode, mini or
not, song, and some other stuff   for triggers (only bg and g color
for now):   a on a placed trigger to edit   left and right on dpad
to change the color to a predefined one from the main levels

that's all I'm gonna give you, work on it with these controls in mind.
(or go the excitebike a to z chunks way idc)
edit: you can use this image from the old repo + from an old commit of the old repo for the color triggers and colors themselves

not planned but might be doable with a lock-on cartridge
more on that in the future

repo: famidash/famidash

license: MIT


text:
Polaris logo replacement

initial logo has black background and is not well suited for a white themed website such as yours
so here's the transparent png version that should look much better
thank you

:+1:

repo: farm-army/farm-army-frontend


text:
Does not infer request property types from schema

Prerequisites

[X] I have written a descriptive issue title
[X] I have searched existing issues to ensure the bug has not already been reported

Fastify version
4.28.1
Plugin version
3.6.0
Node.js version
20.15.0
Typescript version
5.2.2 (also tested with 5.5.4)
Operating system
Linux
Operating system version (i.e. 20.04, 11.3, 10)
archlinux in WSL
Description
I tried following Fastify documentation, but even when copy/pasting their example without any changes, the inferred type of request.query is unknown.
MRE:
const server = fastify().withTypeProvider<TypeBoxTypeProvider>();

server.get("/route", {
  schema: {
    querystring: Type.Object({
      foo: Type.Number(),
      bar: Type.String(),
    }),
  },
}, (request, reply) => {

  // type Query = { foo: number, bar: string }

  const { foo, bar } = request.query;
  // error TS2339: Property 'bar' does not exist on type 'unknown'.
  // error TS2339: Property 'bar' does not exist on type 'unknown'.
});

Link to code that reproduces the bug
No response
Expected Behavior
request.query should be correctly inferred to be of type { foo: string, bar: string }. Same problem with body, response, headers and params types.

same problem

it's Fastify().withTypeProvider... not fastify().withTypeProvider

it's Fastify() not fastify()

If you import it as import * as Fastify from "fastify", you're right.
If you import it as import * as fastify from "fastify", it's fastify().
If you import it as import * as cthulhu from "fastify", it's cthulhu().

Still an issue on v5.1.0.
How is this issue being ignored when it's literally equivalent to "the plugin doesn't work at all"? It doesn't do its only purpose, as a type provider it doesn't provide types.
@Fdawgs

authors: Mahi cesxhin marko1010 paulius-valiunas

repo: fastify/fastify-type-provider-typebox


text:
codes will not login in you get error of "Error logging in user xxxxxx@xxxxxx.com with the given password. Make sure you entered them correctly."

[18:50:34]: ["/Library/Ruby/Gems/2.0.0/gems/fastlane_core-0.12.0/lib/fastlane_core/itunes_connect/itunes_connect_login.rb:41:in login'", "/Library/Ruby/Gems/2.0.0/gems/fastlane_core-0.12.0/lib/fastlane_core/itunes_connect/itunes_connect.rb:62:in initialize'", "/Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/lib/codes/codes_runner.rb:4:in new'", "/Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/lib/codes/codes_runner.rb:4:in run'", "/Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/bin/codes:43:in block (2 levels) in run'", "/Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:178:in call'", "/Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:178:in call'", "/Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:153:in run'", "/Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/runner.rb:428:in run_active_command'", "/Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/runner.rb:68:in run!'", "/Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/delegates.rb:15:in run!'", "/Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/bin/codes:78:in run'", "/Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/bin/codes:85:in <top (required)>'", "/usr/bin/codes:23:in load'", "/usr/bin/codes:23:in <main>'"] [18:50:55]: ["/Library/Ruby/Gems/2.0.0/gems/fastlane_core-0.12.0/lib/fastlane_core/itunes_connect/itunes_connect_login.rb:42:in login'", "/Library/Ruby/Gems/2.0.0/gems/fastlane_core-0.12.0/lib/fastlane_core/itunes_connect/itunes_connect.rb:62:in initialize'", "/Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/lib/codes/codes_runner.rb:4:in new'", "/Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/lib/codes/codes_runner.rb:4:in run'", "/Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/bin/codes:43:in block (2 levels) in run'", "/Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:178:in call'", "/Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:178:in call'", "/Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:153:in run'", "/Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/runner.rb:428:in run_active_command'", "/Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/runner.rb:68:in run!'", "/Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/delegates.rb:15:in run!'", "/Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/bin/codes:78:in run'", "/Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/bin/codes:85:in <top (required)>'", "/usr/bin/codes:23:in load'", "/usr/bin/codes:23:in '"]
[18:50:55]: Couldn't find element '.ng-scope' after waiting for quite some time
/Library/Ruby/Gems/2.0.0/gems/fastlane_core-0.12.0/lib/fastlane_core/itunes_connect/itunes_connect_login.rb:55:in rescue in login': Error logging in user xxxxxx@xxxxxx.com with the given password. Make sure you entered them correctly. (FastlaneCore::ItunesConnect::ItunesConnectLoginError) 	from /Library/Ruby/Gems/2.0.0/gems/fastlane_core-0.12.0/lib/fastlane_core/itunes_connect/itunes_connect_login.rb:40:in login'
from /Library/Ruby/Gems/2.0.0/gems/fastlane_core-0.12.0/lib/fastlane_core/itunes_connect/itunes_connect.rb:62:in initialize' 	from /Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/lib/codes/codes_runner.rb:4:in new'
from /Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/lib/codes/codes_runner.rb:4:in run' 	from /Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/bin/codes:43:in block (2 levels) in run'
from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:178:in call' 	from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:178:in call'
from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:153:in run' 	from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/runner.rb:428:in run_active_command'
from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/runner.rb:68:in run!' 	from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/delegates.rb:15:in run!'
from /Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/bin/codes:78:in run' 	from /Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/bin/codes:85:in <top (required)>'
from /usr/bin/codes:23:in load' 	from /usr/bin/codes:23:in '

I tried using the saved password and also manually entered it in.
neither one worked same error

try
rm /tmp/spaceship*

There is nothing there
Sent from my iPhone5

On Sep 2, 2015, at 12:58 AM, Jerome Lacoste notifications@github.com wrote:
try
rm /tmp/spaceship*
—
Reply to this email directly or view it on GitHub.

@sfransen I had updated the message you replied to and you might not have seen my full answer in your mail.
Try
gem update fastlane_core

Ok I will try that thank again
Sent from my iPhone5

On Sep 2, 2015, at 7:27 AM, Jerome Lacoste notifications@github.com wrote:
@sfransen I had updated the message you replied to and you might not have seen my full answer in your mail.
Try
gem update fastlane_core
—
Reply to this email directly or view it on GitHub.

Thanks I can log on now
but I now getting error  on the Accepting the App Store Volume Custom Code Agreement
from /Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/lib/codes/itunes_connect.rb:48:in run' 	from /Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/lib/codes/codes_runner.rb:4:in run'
from /Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/bin/codes:43:in block (2 levels) in run' 	from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:178:in call'
from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:178:in call' 	from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/command.rb:153:in run'
from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/runner.rb:428:in run_active_command' 	from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/runner.rb:68:in run!'
from /Library/Ruby/Gems/2.0.0/gems/commander-4.3.5/lib/commander/delegates.rb:15:in run!' 	from /Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/bin/codes:78:in run'
from /Library/Ruby/Gems/2.0.0/gems/codes-0.2.2/bin/codes:85:in <top (required)>' 	from /usr/bin/codes:23:in load'
from /usr/bin/codes:23:in `'

@sfransen That's probably because you haven't signed the App Store Volume Custom Code Agreement. Just login on iTunes Connect and try to issue a promo code manually and you should get redirected to the form to sign.

I need this  that was not the problem  but I got it to work now.
Thanks

Cool :+1:

authors: KrauseFx lacostej sfransen

repo: fastlane/codes


text:
Unable to update metadata with deliver. The request could not be completed because: (Spaceship::Tunes::PotentialServerError

New Issue Checklist

[x] Updated fastlane to the latest version
[x] I read the Contribution Guidelines
[x] I read docs.fastlane.tools
[x] I searched for existing GitHub issues

Issue Description
I want update the description and release note of my iOS app on ITC.
On ITC there is already a brand new version that is not submitted yet.

Downloaded the current metadata running fastlane deliver download_metadata
Result:  ✅
Downloaded the current screenshots running fastlane deliver download_screenshots
Result:  ✅
Changed few strings, and tried to upload executing this command:

fastlane deliver run \
  --username #my_username#> \
  --app_identifier #bundleID# \
  --skip_binary_upload true \
  --force true \
  --app_version 8.4.0 \
  --verbose \

or running the same via a lane:
desc "Update metadata"
lane :update_metadata do |options|
  deliver(
    app_identifier: ' #bundleID#',
    username: '#my_username#',
    force: true,
    submit_for_review: false,
    metadata_path: "./metadata",
    skip_screenshots: true,
    verbose: true,
    skip_binary_upload: true,
    app_version: '8.4.0'
    )
end

Expected:
The updated metadata are applied on the release 8.4.0
Current result:
After few invalid_request
[⠙] Activating languages de-DE, en-US... Potential server error received: 'invalid_request'. Retrying after 10 seconds (remaining: 5)...

the script stops with the stacktrace and this log:
         1: from /Users/icalo/.rvm/gems/ruby-head/gems/fastlane-2.107.0/spaceship/lib/spaceship/tunes/tunes_client.rb:436:in `block in update_app_version!'
/Users/icalo/.rvm/gems/ruby-head/gems/fastlane-2.107.0/spaceship/lib/spaceship/tunes/tunes_client.rb:227:in `handle_itc_response': [!] The request could not be completed because: (Spaceship::Tunes::PotentialServerError)

I know this comes from the spaceship error Spaceship::TunesClient::ITunesConnectPotentialServerError, it would be nice to have more info to properly debug the error.
Note: the same is happening also using the spaceship playground
Complete output when running fastlane, including the stack trace and command used

INFO [2018-10-21 16:31:38.01]: Successfully loaded '/Work/****/ios-client/fastlane/Deliverfile' 📄
WARN [2018-10-21 16:31:38.01]: No values defined in './fastlane/Deliverfile'
INFO [2018-10-21 16:31:38.01]: Login to App Store Connect (*******)
Loading session from '/Users/****/.fastlane/spaceship/****/cookie'
Looking for App Store Connect Team with ID *****
INFO [2018-10-21 16:31:40.75]: Login successful
+--------------------------------------+---------------------------+
|                     deliver 2.107.0 Summary                      |
+--------------------------------------+---------------------------+
| username                             | **************            |
| app_identifier                       | com.*****************.ebc |
| skip_binary_upload                   | true                      |
| force                                | true                      |
| app_version                          | 8.4.0                     |
| screenshots_path                     | ./fastlane/screenshots    |
| metadata_path                        | ./fastlane/metadata       |
| platform                             | ios                       |
| edit_live                            | false                     |
| use_live_version                     | false                     |
| skip_screenshots                     | false                     |
| skip_metadata                        | false                     |
| skip_app_version_update              | false                     |
| overwrite_screenshots                | false                     |
| submit_for_review                    | false                     |
| reject_if_possible                   | false                     |
| automatic_release                    | false                     |
| phased_release                       | false                     |
| run_precheck_before_submit           | true                      |
| precheck_default_rule_level          | warn                      |
| ignore_language_directory_validation | false                     |
| precheck_include_in_app_purchases    | true                      |
+--------------------------------------+---------------------------+
INFO [2018-10-21 16:31:41.74]: Making sure the latest version on App Store Connect matches '8.4.0' from the ipa file...
INFO [2018-10-21 16:31:44.55]: '8.4.0' is the latest version on App Store Connect
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/de-DE/description.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/de-DE/keywords.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/de-DE/release_notes.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/de-DE/support_url.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/de-DE/marketing_url.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/de-DE/promotional_text.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/de-DE/name.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/de-DE/subtitle.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/de-DE/privacy_url.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/en-US/description.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/en-US/keywords.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/en-US/release_notes.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/en-US/support_url.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/en-US/marketing_url.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/en-US/promotional_text.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/en-US/name.txt'...
INFO [2018-10-21 16:31:46.08]: Loading './fastlane/metadata/en-US/subtitle.txt'...
INFO [2018-10-21 16:31:46.09]: Loading './fastlane/metadata/en-US/privacy_url.txt'...
INFO [2018-10-21 16:31:46.09]: Loading './fastlane/metadata/copyright.txt'...
INFO [2018-10-21 16:31:46.09]: Loading './fastlane/metadata/primary_category.txt'...
INFO [2018-10-21 16:31:46.09]: Loading './fastlane/metadata/secondary_category.txt'...
INFO [2018-10-21 16:31:46.09]: Loading './fastlane/metadata/primary_first_sub_category.txt'...
INFO [2018-10-21 16:31:46.09]: Loading './fastlane/metadata/primary_second_sub_category.txt'...
INFO [2018-10-21 16:31:46.09]: Loading './fastlane/metadata/secondary_first_sub_category.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/secondary_second_sub_category.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/trade_name.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/first_name.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/last_name.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/address_line1.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/address_line2.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/address_line3.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/city_name.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/state.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/country.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/postal_code.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/phone_number.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/email_address.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/trade_representative_contact_information/is_displayed_on_app_store.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/review_information/first_name.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/review_information/last_name.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/review_information/phone_number.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/review_information/email_address.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/review_information/demo_user.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/review_information/demo_password.txt'...
INFO [2018-10-21 16:31:46.10]: Loading './fastlane/metadata/review_information/notes.txt'...
[⠙] Activating languages de-DE, en-US... Potential server error received: 'invalid_request'. Retrying after 10 seconds (remaining: 5)...
[⠙] Activating languages de-DE, en-US... Potential server error received: 'invalid_request'. Retrying after 10 seconds (remaining: 5)...
[⠙] Activating languages de-DE, en-US...
Looking for related GitHub issues on fastlane/fastlane...
Search query: The request could not be completed because:
invalid_request
URL: https://api.github.com/search/issues?q=The request could not be completed because%3A
	invalid_request+repo:fastlane/fastlane
[⠧] Activating languages de-DE, en-US... ➡  Invalid request with fastlane deliver
https://github.com/fastlane/fastlane/issues/11533 [closed] 1 💬
13 Mar 2018
🔗  You can ⌘ + double-click on links to open them directly in your browser.
Traceback (most recent call last):
22: from /Users//.rvm/gems/ruby-head/bin/ruby_executable_hooks:24:in <main>'         21: from /Users/******/.rvm/gems/ruby-head/bin/ruby_executable_hooks:24:in eval'
20: from /Users//.rvm/gems/ruby-head/bin/fastlane:23:in <main>'         19: from /Users/******/.rvm/gems/ruby-head/bin/fastlane:23:in load'
18: from /Users//.rvm/gems/ruby-head/gems/fastlane-2.107.0/bin/fastlane:23:in <top (required)>'         17: from /Users/******/.rvm/gems/ruby-head/gems/fastlane-2.107.0/fastlane/lib/fastlane/cli_tools_distributor.rb:100:in take_off'
16: from /Users//.rvm/gems/ruby-head/gems/fastlane-2.107.0/deliver/lib/deliver/commands_generator.rb:17:in start'         15: from /Users/******/.rvm/gems/ruby-head/gems/fastlane-2.107.0/deliver/lib/deliver/commands_generator.rb:182:in run'
14: from /Users//.rvm/gems/ruby-head/gems/commander-fastlane-4.4.6/lib/commander/delegates.rb:15:in run!'         13: from /Users/******/.rvm/gems/ruby-head/gems/fastlane-2.107.0/fastlane_core/lib/fastlane_core/ui/fastlane_runner.rb:76:in run!'
12: from /Users//.rvm/gems/ruby-head/gems/commander-fastlane-4.4.6/lib/commander/runner.rb:476:in run_active_command'         11: from /Users/******/.rvm/gems/ruby-head/gems/commander-fastlane-4.4.6/lib/commander/command.rb:153:in run'
10: from /Users//.rvm/gems/ruby-head/gems/commander-fastlane-4.4.6/lib/commander/command.rb:178:in call'          9: from /Users/******/.rvm/gems/ruby-head/gems/fastlane-2.107.0/deliver/lib/deliver/commands_generator.rb:79:in block (2 levels) in run'
8: from /Users//.rvm/gems/ruby-head/gems/fastlane-2.107.0/deliver/lib/deliver/runner.rb:38:in run'          7: from /Users/******/.rvm/gems/ruby-head/gems/fastlane-2.107.0/deliver/lib/deliver/runner.rb:120:in upload_metadata'
6: from /Users//.rvm/gems/ruby-head/gems/fastlane-2.107.0/deliver/lib/deliver/upload_metadata.rb:72:in upload'          5: from /Users/******/.rvm/gems/ruby-head/gems/fastlane-2.107.0/deliver/lib/deliver/upload_metadata.rb:253:in verify_available_languages!'
4: from /Users//.rvm/gems/ruby-head/gems/fastlane-2.107.0/spaceship/lib/spaceship/tunes/app_version.rb:403:in save!'          3: from /Users/******/.rvm/gems/ruby-head/gems/fastlane-2.107.0/spaceship/lib/spaceship/tunes/tunes_client.rb:429:in update_app_version!'
2: from /Users/**/.rvm/gems/ruby-head/gems/fastlane-2.107.0/spaceship/lib/spaceship/tunes/tunes_client.rb:1433:in with_tunes_retry'          1: from /Users/******/.rvm/gems/ruby-head/gems/fastlane-2.107.0/spaceship/lib/spaceship/tunes/tunes_client.rb:436:in block in update_app_version!'
/Users//.rvm/gems/ruby-head/gems/fastlane-2.107.0/spaceship/lib/spaceship/tunes/tunes_client.rb:227:in `handle_itc_response': [!] The request could not be completed because: (Spaceship::Tunes::PotentialServerError)
invalid_request

Environment

✅ fastlane environment ✅
Stack

Key
Value

OS
10.13.6

Ruby
2.6.0

Bundler?
true

Git
git version 2.17.1 (Apple Git-112)

Installation Source
~/.rvm/gems/ruby-head/bin/fastlane

Host
Mac OS X 10.13.6 (17G65)

Ruby Lib Dir
~/.rvm/rubies/ruby-head/lib

OpenSSL Version
OpenSSL 1.1.0h  27 Mar 2018

Is contained
false

Is homebrew
false

Is installed via Fabric.app
false

Xcode Path
/Applications/Xcode-10.0.app/Contents/Developer/

Xcode Version
10.0

System Locale

Variable
Value

LANG
en_US.UTF-8
✅

LC_ALL
en_US.UTF-8
✅

LANGUAGE

fastlane files:
`./Fastfile`
fastlane_require 'fileutils'
fastlane_require 'shellwords'

fastlane_version "1.109.0"
default_platform :ios

desc "Update metadata"
lane :update_metadata do |options|
  deliver(
    app_identifier: 'com.ebaykleinanzeigen.ebc',
    username: 'development@ebay-kleinanzeigen.de',
    force: true,
    submit_for_review: false,
    metadata_path: "./metadata",
    skip_screenshots: true,
    verbose: true,
    skip_binary_upload: true,
    app_version: '8.4.0',
    itc_provider: 'Marktplaats BV'
    )
end

No Appfile found
fastlane gems

Gem
Version
Update-Status

fastlane
2.107.0
✅ Up-To-Date

Loaded fastlane plugins:
No plugins Loaded
Loaded gems

Gem
Version

executable-hooks
1.5.0

bundler-unload
1.0.2

rubygems-bundler
1.4.5

bundler
1.16.6

CFPropertyList
3.0.0

public_suffix
2.0.5

addressable
2.5.2

atomos
0.1.3

babosa
1.0.2

claide
1.0.2

colored
1.2

colored2
3.1.2

highline
1.7.10

commander-fastlane
4.4.6

declarative
0.0.10

declarative-option
0.1.0

unf_ext
0.0.7.5

unf
0.1.4

domain_name
0.5.20180417

dotenv
2.5.0

emoji_regex
0.1.1

excon
0.62.0

multipart-post
2.0.0

faraday
0.15.3

http-cookie
1.0.3

faraday-cookie_jar
0.0.6

faraday_middleware
0.12.2

fastimage
2.1.4

gh_inspector
1.1.3

jwt
2.1.0

memoist
0.16.0

multi_json
1.13.1

os
1.0.0

signet
0.11.0

googleauth
0.6.7

httpclient
2.8.3

mime-types-data
3.2018.0812

mime-types
3.2.2

uber
0.1.0

representable
3.0.4

retriable
3.1.2

google-api-client
0.23.9

json
2.1.0

mini_magick
4.5.1

multi_xml
0.6.0

plist
3.4.0

rubyzip
1.2.2

security
0.1.3

naturally
2.2.0

simctl
1.6.5

slack-notifier
2.3.2

terminal-notifier
1.8.0

unicode-display_width
1.4.0

terminal-table
1.8.0

tty-screen
0.6.5

tty-cursor
0.6.0

tty-spinner
0.8.0

word_wrap
1.0.0

nanaimo
0.2.6

xcodeproj
1.7.0

rouge
2.0.7

xcpretty
0.3.0

xcpretty-travis-formatter
1.0.0

generated on: 2018-10-21

Spaceship error "surfacing" indeed is not very nice and causing problems with debugging actual problems all over the place. Any suggestion how exactly this could be improved? Very open to suggestions here.

authors: ignazioc janpio

repo: fastlane/fastlane


text:
onesignal get [ONESIGNAL APP ID]

How to get [ONESIGNAL APP ID] after run
onesignal(   auth_token: "Your OneSignal Auth Token",   app_name: "Name for OneSignal App",   android_token: "Your Android GCM key (optional)",   android_gcm_sender_id: "Your Android GCM Sender ID (optional)",   apns_p12: "Path to Apple .p12 file (optional)",   apns_p12_password: "Password for .p12 file (optional)",   apns_env: "production/sandbox (defaults to production)" )

Hey @DevFatani ,
you can read a value of ONE_SIGNAL_APP_ID via Lane Context. According to the documentation:

Actions can communicate with each other using a shared hash lane_context, that can be accessed in other actions, plugins or your lanes: lane_context[SharedValues:XYZ].

To wrap it up, you can get the value of ONE_SIGNAL_APP_ID via:
onesignal( auth_token: ... // calling a onesignal action

// after calling onesignal action you can read a value of one signal app id
one_signal_app_id = lane_context[SharedValues: ONE_SIGNAL_APP_ID]

Hope that helped 🙂

authors: DevFatani mollyIV