text:
ByteBuffers/Strings with onNext(T item) and request(int n)?

What type and aggregations should be used to use reactive streams for a byte or character IO?
So for example, if I wanted a Subscriber, does request(n) refer to the number of byte buffers (of unknown size) or to the number of bytes in one or more byte buffers?   What if my subscriber wanted to control both the number of bytebuffers and their total size?
I really don't want to do Subscriber unless there was a way to have an onNext(byte[] item) to receive bytes in bulk.
Ditto for String and characters, or arrays of anything

Hi @gregw,
demand-element relation is explained here: https://github.com/reactive-streams/reactive-streams-jvm#subscriber-controlled-queue-bounds
Sadly, Java (or the JDK rather) does not have a standard representation for an immutable chuck of bytes (just as String is a representation for an immutable chunck of unicode code points). An acceptable solution for your use case may be to send asReadOnly() versions of ByteBuffer?

Hi @viktorklang ,
Thanks for the response... firstly is this the right place to discuss this kind of thing.  Please point me to the appropriate forum if not.
The link you gave makes it clear that the value passed in request(int n) will translate to the invocation count of onNext(), which is exactly the problem I don't know how to get around?
If I have a Subscriber and do a request(1), then I do know that onNext() will be called once and only once, but I don't know if it will be called with a ByteBuffer that contains a single byte or one that contains 4GB!   Considering that onNext has to fully consume the content passed, that is a big problem.
What we are trying to do is to provide the Flow semantics directly from the Jetty servers asynchronous IO streams, so that developers don't have to deal with the complexities of Servlet API's async callbacks.

I put together the https://github.com/ReactiveX/RxJavaString/ project to help with some of the issues with splitting and encoding across different chunks of UTF8 bytes.  Not sure any of it would be directly useful for ByteBuffers.  If you want maybe that project would be a good place for a ByteBuffer specific operations if not then maybe it could be used as a template for your own custom operators.

George,
I can see that such concerns as splitting/encoding of byte streams that
contain UTF8 characters will indeed be important for solutions in this
space.    I'm new to these APIs but my initial thought is that I would
expect to see something like:
class ITF8Processor implements Flow.Processor<ByteBuffer,CharSequence>
which could be used to convert bytes collection to characters sequences and
such a class would indeed need to handle the problem of multi-byte
characters splitting over byte buffers.
However, looking at your code I can't see that you've addressed the issue
of what unit to express back pressure control in. I can see your
onNext(byte[]) method doing the conversion, but I can't see it calling
Subscriber.request to communicate that it is able to receive more bytes?
It is that question I need resolved so I can get over the initial hump and
start using these APIs and perhaps trying out things like integration to
RxJava.
cheers
On 29 May 2015 at 12:44, George Campbell notifications@github.com wrote:

I put together the https://github.com/ReactiveX/RxJavaString/ project to
help with some of the issues with splitting and encoding across different
chunks of UTF8 bytes. Not sure any of it would be directly useful for
ByteBuffers. If you want maybe that project would be a good place for a
ByteBuffer specific operations if not then maybe it could be used as a
template for your own custom operators.
—
Reply to this email directly or view it on GitHub
https://github.com/reactive-streams/reactive-streams-jvm/issues/270#issuecomment-106663715
.

--
Greg Wilkins <gregw@webtide.com gregw@intalio.com>  - an Intalio.com
subsidiary
http://eclipse.org/jetty HTTP, SPDY, Websocket server and client that scales
http://www.webtide.com  advice and support for jetty and cometd.

You'll either have to have the Producer generate reasonable sized ByteBuffers or have an intermediate step that slices them into smaller parts.
Might I also suggest you take a look at https://github.com/ReactiveX/RxNetty.

Ah I've seen that this has been briefly discussed before in https://github.com/reactive-streams/reactive-streams-jvm/issues/47 and closed because it was too hard to solve simply!
Sorry but that seams like a bit of a cop out to me and will make the most simple of processors impossible to write asynchronously.
For example, if I want to write a processor the does UTF9 conversion of ByteBuffer to CharSequence, then it is possible for onNext to be called with a massive memory mapped file as a ByteBuffer, which will never ever fit into memory.  So the processor will have no choice but to block and slice off a buffer of characters at a time, feeding them to its downstream subscriber until all have been consumed.
It is not even possible to write a non-blocking fragmenting Processor whose only role is to chop a big buffer into smaller ones that might avoid this problem downstream.
I don't think this is not a too hard problem and I can see two basic approaches:

do a bit of work around defining ownership of an Item beyond the return of onNext(). ie if a processor was able to hold onto a reference to ByteBuffer item and return from onNext(), then at least asynchronous processing could be achieved.

Add a request(long items, long size) method to the Subscription with the semantics that a call to request(long n) is equivalent to request(items,Long.MAX_VALUE).     There would also need to be a generic way of determining the size of an arbitrary T item.   If a producer had an Item that was too large for the requested size, it could either: wait for more size to be requested; slice the item if it knows how; or simply call onError(new InsufficientSizeException).

It would be a real pity if this semantic is put into the too hard basket.  The whole point of APIs like this is that async is hard and needs to be simplified.  Having a single async Flow abstraction that can handle bytes, chars, frames, objects, more complex objects etc.   while keeping the same semantic and backpressure mechanism would be awesome!

@abersnaze
Surely at one end of a Producer chain their might be a user who is not going to be constrained by reasonable sizes.  They are just going to memory map the biggest file they can find as a ByteBuffer and feed it into a Flow and want it to work.
You can write a Processor that slices large items into smaller items, but I can't see how it can be written asynchronously because the large item will arrive in an onNext call which can't return until the entire item has been consumed.
OK so maybe this can be done in a Producer, but to do so would require another asynchronous API and isn't the whole point of this exercise to give a nice portable async reactive API that users can know love and use?    If we have to switch to another API when we hit uncontrolled user code, then it kind of defeats the point! No?
Also, even if the initial Producer creates right sized chunks, who is to say that they are right-sized all the way through a chain of Producers and Subscribers.  If you are to use the Processor model then it needs to be able to asynchronously slice big items into small items.  The current API does not allow this in a non blocking way, so the model will fail for an arbitrary chain of Processors with differing size limitations within the chain.

Here are two examples of splitting potentially oversized byte[]/ByteBuffers into smaller ones:
public class ByteArraySplitting {
    public static void main(String[] args) {
        int maxSize = 300;

        Observable.just(new byte[1024]).repeat(2)
        .flatMap(a -> {
            int capacity = (a.length + maxSize - 1) / maxSize;
            List<byte[]> chunks = new ArrayList<>(capacity);
            int i = 0;
            int remaining = a.length;

            while (remaining != 0) {
                int len = Math.min(maxSize, remaining);
                byte[] chunk = Arrays.copyOfRange(a, i, i + len);

                chunks.add(chunk);
                i += len;
                remaining -= len;
            }
            return Observable.from(chunks);
        })
        .subscribe(b -> System.out.println(b.length));

        ByteBuffer bb = ByteBuffer.allocate(1024);
        bb.put(new byte[1024]);
        bb.flip();

        Observable.just(bb)
        .flatMap(a -> {
            int capacity = (a.limit() + maxSize - 1) / maxSize;
            List<ByteBuffer> chunks = new ArrayList<>(capacity);
            int i = 0;
            int remaining = a.limit() - a.position();

            while (remaining != 0) {
                int len = Math.min(maxSize, remaining);
                ByteBuffer chunk = a.slice();

                chunk.position(i);
                chunk.limit(i + len);

                chunks.add(chunk);
                i += len;
                remaining -= len;
            }
            return Observable.from(chunks);
        })
        .subscribe(b -> System.out.println(b.position() + " .. " + b.limit()));
    }
}

Cutting down on the size is straighforward, but if you want to combine smaller chunks up to the limit and split larger chunks down to the limit, that is more complicated.

@akarnokd
I can't relate the examples you've give to an asynchronous implementation of a Flow.Processor<ByteBuffer,ByteBuffer>, where the big buffers will arrive in an onNext(ByteBuffer item) call and have to be fed to another Subscriber, who may only have done a request(1) and another request call may be some time in coming.   Thus once the big buffer has been split, the processor can only calls its subscribers onNext(ByteBuffer) with a small buffer and then has to wait until another call to request(n) is made - hence it becomes blocking?

In RxJava all of the non blocking backpressure logic is in the last line return Observable.from(chunks); The chunks are computed eagerly but they are parcelled out as request(n) calls come in.
Here is the implementation of the from
    private static final class IterableProducer<T> implements Producer {
        private final Subscriber<? super T> o;
        private final Iterator<? extends T> it;

        private volatile long requested = 0;
        @SuppressWarnings("rawtypes")
        private static final AtomicLongFieldUpdater<IterableProducer> REQUESTED_UPDATER = AtomicLongFieldUpdater.newUpdater(IterableProducer.class, "requested");

        private IterableProducer(Subscriber<? super T> o, Iterator<? extends T> it) {
            this.o = o;
            this.it = it;
        }

        @Override
        public void request(long n) {
            if (REQUESTED_UPDATER.get(this) == Long.MAX_VALUE) {
                // already started with fast-path
                return;
            }
            if (n == Long.MAX_VALUE) {
                REQUESTED_UPDATER.set(this, n);
                // fast-path without backpressure
                while (it.hasNext()) {
                    if (o.isUnsubscribed()) {
                        return;
                    }
                    o.onNext(it.next());
                }
                if (!o.isUnsubscribed()) {
                    o.onCompleted();
                }
            } else if(n > 0) {
                // backpressure is requested
                long _c = BackpressureUtils.getAndAddRequest(REQUESTED_UPDATER, this, n);
                if (_c == 0) {
                    while (true) {
                        /*
                         * This complicated logic is done to avoid touching the volatile `requested` value
                         * during the loop itself. If it is touched during the loop the performance is impacted significantly.
                         */
                        long r = requested;
                        long numToEmit = r;
                        while (it.hasNext() && --numToEmit >= 0) {
                            if (o.isUnsubscribed()) {
                                return;
                            }
                            o.onNext(it.next());

                        }

                        if (!it.hasNext()) {
                            if (!o.isUnsubscribed()) {
                                o.onCompleted();
                            }
                            return;
                        }
                        if (REQUESTED_UPDATER.addAndGet(this, -r) == 0) {
                            // we're done emitting the number requested so return
                            return;
                        }

                    }
                }
            }

        }
    }

@abersnaze OK I'm going to need to play with your code a bit more to see if I can understand how you are avoiding blocking.   Is this code against:
<dependency>
	<groupId>com.netflix.rxjava</groupId>
	<artifactId>rxjava-core</artifactId>
	<version>0.20.7</version>
</dependency>

@gregw You are better off with the latest:
<dependency>
    <groupId>io.reactivex</groupId>
    <artifactId>rxjava</artifactId>
    <version>1.0.11</version>
</dependency>

@abersnaze's example seems to be an older one because it has a concurrency bug in its fast-path.

Thanks @akarnokd and @abersnaze, I've got the code running and understand it better (I'm still getting my head around that code style).
I can see that example can be kind of thought of as asynchronous, as it holds the chunks until a request(n) call comes in from a subscriber, in which case it hands out n chunks.   If the subscriber was remote or truly asynchronous (which it is not in this example), then there would be no thread waiting to hand out the next chunk.
However, this is essentially just a producer that slices a big buffer into lots of smaller buffers and hands them out as requested.   This is not the Flow.Processor case that I'm concerned with where the big buffer has itself been delivered by an asynchronous callback.
Firstly pre computing the chunks may not always be possible.  OK for ByteBuffers that can be sliced, but if it was being converted to a CharSequence, then precomputing would triple the memory commitment, which is kind of against the whole purpose of slicing it up in the first place.
Even if you solve this by taking a single slice as demanded, you still need to hold onto the source ByteBuffer while all the slices are taken.  If that source ByteBuffer has been provided by a onNext call from a truly async source, then I don't see how this can work.  It works in this example because the ByteBuffer is not recycled/reused for the duration of the test.  But in a real usage, the ByteBuffer passed in an onNext call is likely to be reused immediately after the onNext returns - thus any slices taken subsequently will point to the wrong data.
Anyway, let me get my head a bit more into this style of code and I'll write a unit test that demonstrates the async semantic I'm trying to achieve... this could take me a few days.

In Rx we like to use Observable.from because it hides some complicated logic but it requires data to be precomputed. We've been experimenting with an AnstractOnSubscribe that hands the back pressure and delegates to a closure to produce just enough data when needed.

@abersnaze It is not so much the use of Observable.from that is the issue for me.  If the task is slicing a ByteBuffer into small slices, then it is fit for purpose.       The problem is not preslice or slice as you go, the problem is the scope of the big buffer that you are slicing from.   If that big buffer is passed in from an onNext call as part of a chain of processors, then that onNext call cannot return while slices are current (as it backs them) or there is more slicing to do.    Thus that onNext call becomes blocking not asynchronous.

You can apply observeOn before the flatMap or subscribeOn after from so the stream gets asynchrony and your source is not held back that much.

Sorry for the late reply, I didn't have the cycles to read through the whole thread, so forgive me if this has already been said:
There is absolutely no problem going from Collection as element type to T, nor the other way around,
demand is signalled on a per-element basis so that if you want to limit the size of the Collection then you should expose that when creating the Publisher:
someSource.strictChunkedPublisher(size = N).subscribe(subscriber)

the reason for this is simple—since the Collection has already been created, it is not occupying variable subscriber buffer space.
Perhaps you want to allow for "read as much as you have but don't send me more than N per chunk":
someSource.lenientChunkedPublisher(size = N).subscribe(subscriber)

The reasons for not messing around with specifying byte sizes are, but not limited to:
A) Given a type T what is its size?
B) Given type T and a max size of N, what do you do with Ts larger than N?
C) Do you allow for sending less than N?
D) What happens with the last data if it is less than N?
E) What does "demand N of size X" mean in practice?
…
Given that we can create types that reflect intent. (You could create a ByteBuf256 if you want to be explicit about capacity and optimization opportunities).
Given that we can transform chunks into their contents in a non-blocking and backpressured way (by peeling off elements and emitting their contents as requested)
Given that we can take elements and chunk them. (classing windowing)
===>
What would, if any, be the benefits of changing to include for chunking.
(Benchmarks are encouraged)
(Worth noting is that currently the RS spec allows for implementations that push ~175 million elements a second (point-to-point).)

There was a previous discussion that might also be of interest: #47

FWIW we're dealing with this issue in the reactive-ipc project. [1]
The problem really becomes one of translation. By that I mean: Subscriber has capacity for X bytes but can't request bytes directly from the Publisher, only buffers. That means the Subscriber has to communicate to the Publisher (dynamically and not at CTOR/creation time) its current capacity in a unit that the Publisher understands but does not overrun the capacity of the Subscriber.
In short, I think there has to be additional information similar to what TimeUnit does for the long associated with time. Either a DemandUnit type abstraction or some other way to set an additional limit other than just the long associated with request(long).
[1] - https://github.com/reactive-streams/reactive-streams-jvm/issues/270

@viktorklang
I understand that this is not an issue in a Publisher and it can fragment a large collection it already has in memory easy enough.   The use-case I'm concerned with is if the memory limitation is in a Processor.     Two examples come to mind:

an async output stream is internally implemented with ReactiveStreams that use Processers to send the stream over a HTTP2 connection.  So you will have Processors that compress the data, fragment it into HTTP2 frames and yet another Processor that TLS encrypts those frames.  At each of these steps there will be either a maximum or optimum fragment size to deal with for compression, protocol frame size or encryption processing and the origin Publisher does not know what they are and has no means of discovering them.     If the protocol frame size is smaller than the size of data published, then the Processors will have to block until they get no back pressure from their subscribers.

To illustrate this problem I've written an example (against the reactive streams API rather than rxJava), that includes a long README about the problems I can't get my head around:
https://github.com/jetty-project/jetty-reactive/tree/FragmentingProcessor
Again it is the Processors that I'm most concerned with. The origin publisher and ultimate subscriber have a lot more flexibility to deal with these kinds of issues.   If an ecosystem of utility processors is to be developed (eg compressors, encryptors, encoders etc.) then they need to be able to operate asynchronously within the API provided.   That is what I'm not currently seeing.

@rkuhn yes #47 is essentially the same issue.... problem is that it was closed as being too hard to fix!
I'm evaluating this API against a real asynchronous use-case that is in need of a better API (Servlet async IO) and I think it reasonable that an API proposed for java 9 can handle such use-cases at least as good as callbacks.    Hence I'd like to open the discussion again to see if there are techniques within the API as proposed that can help or if there are some possible changes to the API.  As without one or the other, reactive streams do not look suitable for these use-cases.
See the README in https://github.com/jetty-project/jetty-reactive/tree/FragmentingProcessor for more detailed description of the use-cases I'm concerned with.
cheers

@viktorklang ooops I didn't give my second example.... never mind, it is the same as the first one.

@jbrisbin

The problem really becomes one of translation. By that I mean: Subscriber has capacity for X bytes but can't request bytes directly from the Publisher, only buffers. That means the Subscriber has to communicate to the Publisher (dynamically and not at CTOR/creation time) its current capacity in a unit that the Publisher understands but does not overrun the capacity of the Subscriber.

The memory has already been allocated for the chunk, so I don't see how the Subscriber's buffer would be overrun? (The Subscriber's buffer is in Ts, no matter if T is composite or not)

@viktorklang Surely you don't think the only significant measure of a subscriber's capacity is an often unmeasurable (at the least unknowable) impact on RAM usage? There are other kinds of constraints that have nothing to do with RAM usage.

@jbrisbin

@viktorklang Surely you don't think the only significant measure of a subscriber's capacity is an often unmeasurable (at the least unknowable) impact on RAM usage? There are other kinds of constraints that have nothing to do with RAM usage.

Hmm, how do you mean in this case?
To illustrate my point, let's say we have a ByteBuf256 type which, unsurprisingly, holds up-to 256 bytes.
Now, as a Subscriber<ByteBuf256> the question is what your demand pattern ought to look like: if you only want to buffer at most 256 bytes you issue a demand of 1 and wait for the next piece of data before issuing more. I.e. your input buffer is of size 1 and your element type can carry at most 256 bytes.
Am I missing something?

Actually.... I think I'm about to answer my own question.....
A Processor that receives a ByteBuffer in onNext that is larger than it can process without blocking (do to lack of backpressure) can just keep a reference to it and return from the onNext call... provided that:

It didn't have more than request(1) outstanding, so the Publisher will not call onNext again
The Publisher understands the contract that just because onNext() returns does not mean that the passed buffer has been fully handled and can be reused.
The Publisher will wait (threadlessly) for a subsequent request(1) to inform it that processing of it's last onNext has now completed and the passed buffer may be recycled.
The Publisher will wait (threadlessly) for a request(1) before call onCompleted() so that it does not try to complete while a previously passed buffer is still being processed.

Hmmm  I'll give that a go in my repo tomorrow (1am here now) and report back if that approach solves my issues.

@gregw
I'm opportunistically answering your answer instead of your previous comments (let me know if I should)

Actually.... I think I'm about to answer my own question.....

A Processor that receives a ByteBuffer in onNext that is larger than it can process without blocking (do to lack of backpressure) can just keep a reference to it and return from the onNext call... provided that:

It didn't have more than request(1) outstanding, so the Publisher will not call onNext again

If a processor decides to issue more demand upstream than is signalled downstream, it (by design) needs to be willing to buffer that itself. This does not entail any blocking in the "parking a Thread"-sense of the word.

The Publisher understands the contract that just because onNext() returns does not mean that the passed buffer has been fully handled and can be reused.

Exactly. Think of onNext as sending the element asynchronously to the Subscriber, it could technically "live" on another machine. This is why the term signal is used in the spec.

The Publisher will wait (threadlessly) for a subsequent request(1) to inform it that processing of it's last onNext has now completed and the passed buffer may be recycled.

No, this is faulty. request is never to be assumed to be an Ack. As per the spec the Subscriber is free to issue requests whenever it likes. (barring things like cancellation, onComplete/onError etc).

The Publisher will wait (threadlessly) for a request(1) before call onCompleted() so that it does not try to complete while a previously passed buffer is still being processed.

See previous answer.

Hmmm I'll give that a go in my repo tomorrow (1am here now) and report back if that approach solves my issues.

What you'll need to do in this case is to control the pipeline:
YourPublisher -> CustomerProcessor<ByteBuffer, ByteBuffer> -> YourSubscriber
So you can collect the ByteBuffers as they are passed through.
Keep in mind that anything in between could in theory retain a reference to your buffer and mess around with it after they should.
Does that help?
Let me know if there's anything I can clarify further or if I have misunderstood something.

@viktorklang regarding your comment to @jbrisbin
Consider a GzipInflatorProcessor implements Processor<byte[],byte[]>
It's subscriber calls request(1) on the inflator, so it calls request(1) on it's publisher, which responds by calling inflator.onNext(byte[]) with a 256 byte array.
But it is an inflator, so that 256 byte array can expand to an almost arbitrary large byte[].    There are several possibilities:

inflator just inflates to a single huge byte[] and has unlimited memory commitment.
inflator interates inflating chunks of bytes[] and blocks while waiting for request(n) calls so that it can call onNext with each chunk on its subscriber
inflator returns from onNext without having completely inflated the 256 bytes.  The processor knows that it cannot mess with that byte array until it sees a request(n>0) from the inflator (as per my previous comment)

@gregw FWIW in Reactor we had to implement his "remainder" functionality for our Codec facility. That sounds very similar.

@viktorklang replying to your message that crossed mine.
See my inflator example to understand why I'm concerned with collecting buffers as they pass through. That can result in an arbitrary memory commitment.
Surely a Processor is a Subscriber, so it is free to without its subsequent request(n) calls until it has finished handling a previous onNext call.
I understand that using a request(1) as an ack is not exactly correct, but I do not see how else a publisher can recycle a buffer it has previously passed to onNext.    It's not acceptable to say that it can't recycle it as servers will just not scale if they create that kind of garbage for each message.
So without the request as ack, how else can a Publisher (or Processer) recycle a buffer?

@gregw
@viktorklang regarding your comment to @jbrisbin

Consider a GzipInflatorProcessor implements Processor

It's subscriber calls request(1) on the inflator, so it calls request(1) on it's publisher, which responds by calling inflator.onNext(byte[]) with a 256 byte array.

But it is an inflator, so that 256 byte array can expand to an almost arbitrary large byte[]. There are several possibilities:

inflator just inflates to a single huge byte[] and has unlimited memory commitment.
inflator interates inflating chunks of bytes[] and blocks while waiting for request(n) calls so that it can call onNext with each chunk on its subscriber
inflator returns from onNext without having completely inflated the 256 bytes. The processor knows that it cannot mess with that byte array until it sees a request(n>0) from the inflator (as per my previous comment)

I'll try simplify the example:
Let's say that you have a processor that given a T can produce an unbounded number of Ts.

Now, if the routine that possibly creates an unbounded number of Ts doesn't have a means to control the number of Ts it produces, nothing Reactive Streams can do will ever be able to fix that. It is equivalent to a Publisher that cannot be controlled-> you'll have to decide to buffer and drop.

@viktorklang replying to your message that crossed mine.
See my inflator example to understand why I'm concerned with collecting buffers as they pass through. That can result in an arbitrary memory commitment.

Surely a Processor is a Subscriber, so it is free to without its subsequent request(n) calls until it has finished handling a previous onNext call.

Sure, but only if you control the implementation, i.e. you know more about the pipeline than the RS spec can know. I'm talking from the general perspective of what you can and can't assume wr.t. arbitrary Publishers, Subscribers and Processors given the spec and tck.

I understand that using a request(1) as an ack is not exactly correct, but I do not see how else a publisher can recycle a buffer it has previously passed to onNext. It's not acceptable to say that it can't recycle it as servers will just not scale if they create that kind of garbage for each message.

So without the request as ack, how else can a Publisher (or Processer) recycle a buffer?

Safest strategy is to add a collector stage at the end that ferries the buffer back to the publisher.
Does that make sense?

@viktorklang
I think a collector that ferries the buffer back to the publisher is also dependent on controlling the implementation beyond what the RS spec tells you.  You wont be able to develop utility processors as they will not know how to return the buffers passed to them from arbitrary publishers (which may be removed by several processors).
However, it is probably a better solution than the request as ack idea as it at least allows for request(n>1) to be sent to the Publisher to avoid possible latency issues.
I'll experiment further.

@gregw
You'll be able to hook in any processor you like as long as you control where the buffers are injected and where they are collected. Or?
Let me know what your experimentation reveals, this is an interesting topic.
Looking forward to see what comes out of it.

OK I think I'm understanding RS better and will close this issue.
The problem that I was having is that I wanted to implement a Processor that was async, bounded memory and acknowledging (so callers could recycle buffers).    I've worked out that you can have only 2 of those 3 attributes.  I've been able to implement an async & bounded memory process so long as I don't attempt to use the return of onNext or a subsequent request(n) call to indicate acknowledgement (so that buffers can be recycled or the demo exit after completion).
You can see the results of this revelation here.
While I think acknowledgement is an important attribute, it is not the subject of this issue and I'll move the discussion of it to the list.   My demo achieves acknowledgement only by going outside the standard RS API.
Two other things I'll note of interest here:  Firstly the demo contains an abstract IteratingProcessor that is based on the Jetty IteratingCallback.  This is a useful pattern to avoid arbitrarily deep recursion (onNext(item) calls request(n) calls onNext(item) calls request(n)....) which is a common problem with async frameworks.
The other is that I've noticed that RS are similar to a scheduling strategy that we've implement in Jetty called Eat What You Kill.   EWYK achieves mechanical sympathy by implementing the ethos of a thread should not produce a task unless it is able to immediately consume it. This avoids queues, dispatch delays and allows tasks to be consumed with hot tasks.
RS style encourages similar mechanical sympathies by requiring publishers to not publish until there is demands that originates from the ultimate subscriber. This can allow a single thread to call through a chain of onNext calls, keeping it's cache hot while avoiding queues and dispatch delays.

@gregw
Just passing by and looking at your problem. What would be if you copied the 256 bytes buffer when you received it in your GzipInflatorProcessor to a private buffer and then decompress incrementally from the private copy when the downstream requests more. So you don't retain a reference to buffer passed to you. Surely this will mean more memory copying, but that's an alternative if you don't want to manage byte[] arrays ownership. You treat byte[] arrays as value types. Does that make any sense?

Hi @gregw,
I can't see any tests in the project—I'd strongly recommend applying the TCK to your processor, it will assist you in ironing out potential spec violations.
Speaking of the spec, I'd love to get your feedback on it, is there things that need improvement, what was good etc?

@plevart Sure copying data get's around some of the issues and copying 256 bytes in that example is not a big deal.  But copying data passed in that can be of arbitrary size is not a good general solution as it is hard to enforce memory bounds.
cheers

@viktorklang Thanks for the suggestion of applying the TCK.  I think it is probably a bit early for our experiments, but we will certainly look at it soon.
With regards to the spec, it is also a bit early for me to say anything too definitive... specially as I think I've just spent the best part of a week barking up the wrong tree with it :)   Although I guess that might mean that some of the documentation and examples could be improved.
But now that I've adjusted my head to think more in line with how the API is intended to be used, I can give you some initial feedback.    It is definitely a lot simpler to work with than other async APIs, so while I think there is a lot of good in the Servlet async IO API, it is a very tricky API to use and we are very interested in wrapping it as RS's to make it easier to use.  That is where our focus has shifted in our experimental repo.
The other thing I like is the mechanical sympathy that results from RS, with a single thread able to call all the way through a chain of onNext calls with a hot CPU cache and no dispatch.  The IteratingProcessor model we've implemented should be good at stopping any recursion problems and is much simpler to use than the IteratingCallback we use with our callback model.
However, we still do have some concerns with the lack of acknowledgement, so a publisher does not know if it can recycle an item, can't get exceptions in some circumstances and doesn't know if an onCompleted() has progressed to completion all the way through a chain.
But give us a few more days to get our heads more into it and also try out some more real use cases, perhaps with some other implementations.  But so far its pretty good!
cheers

@gregw
I assume you saw these? https://github.com/reactive-streams/reactive-streams-jvm/tree/master/examples/src/main/java/org/reactivestreams/example/unicast
You may want to inspect other implementations (implementing it is non-trivial just because the problem has inherent complexity) but the end-user will not be implementing it themselves but use an implementation. It may also be interesting for you to leverage an already existing implementation to prototype with.
Looking forward to your feedback.

authors: abersnaze akarnokd gregw jbrisbin plevart rkuhn viktorklang

repo: reactive-streams/reactive-streams-jvm


text:
Add some documentation for testing

I'm trying to use this to propagate context but I can't write a test to validate it works. Can you add some snipped showing how to do this? Thanks

The goal of this project is to propagate context for logging tools, not a general purpose context propagation tool. Can you elaborate about your use case?

I'm using it to propagate headers from service to service in a cleaner way, I made a POC and works fine but I can't make tests to be sure.

In your test, you could implement a server that sits between a client and a backend server. The client sends a request with a HTTP header and you verify it is sent to the backend.
Perhaps you can inspire from https://github.com/reactiverse/reactiverse-contextual-logging/blob/main/src/test/java/io/reactiverse/contextual/logging/ContextualLoggingIT.java

source: gharchive/issue

authors: mreparaz tsegismont

license_type: permissive

license_source: github-api


text:
Unable to load one or more of the requested types.

This runtime error occurs after ading the optimization package.
link

Try looking at the LoaderExceptions property on the exception to see what the actual error message is.

Closing this out as I haven't heard back, but please feel free to reopen this issue if you're still encountering problems.

@Daniel15 same error here. I made a new solution following the MVC5 tutorial, and the error didn't come up. I fixed it in the old one by upgrading to ReactJS.NET v3.0 . There doesn't seem to a way of accessing LoaderExceptions property from our code, it's some pre-application start stuff handled somewhere in the react dll, probably here: React.Web.WebInitializer.Initialize()

authors: Daniel15 damiangreen simbs

license: MIT


text:
GitHub suggestion bug

Galera, só pra compartilhar com vocês:
Pessoal que está revisando, perceberam que o suggestion do GitHub as vezes não está fazendo o highlight correto das diffs?
Estou revisando esse PR: https://github.com/reactjs/pt-BR.reactjs.org/pull/72/files
E se vc olharem, algumas linhas não aparecem a diff na sugestão. Veja a linha 64, por exemplo: https://github.com/reactjs/pt-BR.reactjs.org/pull/72/files#r255440979
Isso tá acontecendo com vocês também?
Sabem se tem algum lugar que podemos reportar o bug? Eu não achei nada de muito diferente nessas linhas que pudesse estar causando isso, o único padrão que parece emergir é que todas são relativamente longas (ficam com mais de 6 linhas na interface).

Já tentou enviar esse report para o GH via twitter? @halian-vilela

Já tentou enviar esse report para o GH via twitter? @halian-vilela

Ainda não! Mas é uma boa ideia. Vou enviar lá mais tarde! Valeu!
Mas chegou a acontecer com vocês?

@halian-vilela todavia não.

Nunca aconteceu comigo também. Podemos fechar este issue? @halian-vilela

Pode fechar @eduardomoroni, ainda está rolando, mas de fato tenho que enviar lá pro pessoal do GH, vou ver se faço isso hj mais tarde...
Valeu!

authors: eduardomoroni glaucia86 halian-vilela


text:
[Feature Request] Use indexRedirect in route's configuration object

Hello,
I tried to use indexRedirect in a route configuration object, but it seems that it's ignored.
Would it be possible to add it with a syntax like :
indexRedirect: { to: "path" },

Thanks to all the contributors for this awesome library.

Why was this closed? It's a legitimate feature request @taion.

Does this not work? https://github.com/reactjs/react-router/issues/2256#issuecomment-148249068 (I don't have a test case in front of me, so it's a legitimate question)

I'm sorry,
it seems that my issue is a duplicate of https://github.com/reactjs/react-router/issues/2559
I assumed there were a feature parity between JSX / plain object notation, but apparently I was wrong.

It works just fine; that's just not the syntax.

@taion Could you point me to the doc showing the right syntax please ?

No – the issue tracker is not the place for support questions.

@jgoux Click the link in my comment right above yours...

@timdorr This doesn't work for me, I also have an infinite redirect loop, like the last comment on the issue : https://github.com/reactjs/react-router/issues/2256#issuecomment-151993906
The route in question :
import { view } from "./view"
import { update } from "./update"
import { route as rechercheRoute } from "./components/Recherche"
import { route as saisieRoute } from "./components/Saisie"

export const route = {
  component: view,
  reducer: update,
  onEnter: (_, replaceState) => { replaceState(null, saisieRoute.path) },
  childRoutes: [
    rechercheRoute,
    saisieRoute
  ]
}

I assume I have to find the way to redirect only when no child is matched (this is what IndexRoute does).
Sorry to bother you all, I'll continue on stackoverflow/discordapp instead,
have a nice day ! :+1:

authors: jgoux mrtnbroder taion timdorr

repo: reactjs/react-router

license: mit

license_source: bigquery


text:
Server Rendering documentation can't return the static bundle.js

Do you want to request a feature or report a bug?
bug
(If this is a usage question, please do not post it here—post it on Stack Overflow instead. If this is not a “feature” or a “bug”, or the phrase “How do I...?” applies, then it's probably a usage question.)
What is the current behavior?
When '/static/bundle.js' is looked up from the client, the handleRender function is called returning the index.html
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem via https://jsfiddle.net or similar.
Because of app.use(handleRender); , the index is always returned.
What is the expected behavior?
The index.html containing the  with his appropriate state, should only be returned when the user enters the site for the first time.
Which versions of Redux, and which browser and OS are affected by this issue? Did this work in previous versions of Redux?
N/A, all, all

Added pull request : https://github.com/reactjs/redux/pull/2265

repo: reactjs/redux


text:
real-world example does not work with node 4(update README?)

Do you want to request a feature or report a bug?
bug
Hi! the real-world example will not work with node 4. Because of related issue in react-scripts Had to update to 6.0.0 on Mac machine. Maybe it is obvious but I had to spend some time to get it. Is it possible to update readme for example? I can volunteer for pull request if that's ok.
(If this is a usage question, please do not post it here—post it on Stack Overflow instead. If this is not a “feature” or a “bug”, or the phrase “How do I...?” applies, then it's probably a usage question.)
What is the current behavior?
example does not work with node 4.
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem via https://jsfiddle.net or similar.

clone redux repo
navigate to real-world example folder
npm install
npm run
error appears

/node_modules/react-scripts/scripts/start.js:33

const {

  ^

SyntaxError: Unexpected token {

What is the expected behavior?
example is being served at localhost
Which versions of Redux, and which browser and OS are affected by this issue? Did this work in previous versions of Redux?
Issue appear on master redux branch

We could add engine lines to all the projects' package.json files. But the incompatibility is upstream from this project anyways. They should similarly add that line so it won't attempt to run on old versions of node. Unfortunately, that's really out of our hands.

authors: timdorr wolendranh


text:
Test to ensure inputSelectors called with same params as the selector

On master, the inputSelectors are called with an extra undefined param when the selector is only called with the state. This is usually not a problem, but in my specific case, some inputSelectors are curried functions, and in the result func, I use the returned function. I know it kind of defies the concept of createSelector, but in these cases, perf does not matter, and it lets me use the same API/patterns consistently in my app.
This bug is due to the props param (possibly undefined) passed to dependency, on this line.
I was planning to submit a fix, however, the 3.0.0 rewrite did fix this bug! So here is a test to prevent any regressions on this side :) If you add this test to master, you'll see that it breaks.
I can also submit this (with the corresponding fix) to the current master branch, if you think 3.0.0 won't be publish soon. I have no emergency on this, I can use a simple workaround on my side, it just makes my code a bit less readable!

Coverage remained the same at 100.0% when pulling a2faebb8276c8609eb8d931281ddcb5bc5ea8de9 on AugustinLF:tests/ensure-no-undefined-params into 2abc7d032f89e7d04afabf7b4c7dfad26939e965 on reactjs:3.0.0.

Codecov Report

Merging #226 into 3.0.0 will not change coverage.
The diff coverage is n/a.

@@         Coverage Diff          @@
##           3.0.0   #226   +/-   ##
====================================
  Coverage    100%   100%
====================================
  Files          1      1
  Lines         15     15
====================================
  Hits          15     15

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2abc7d0...a2faebb. Read the comment docs.

I've published 3.0.0 now, so I guess we can close this? (Thanks very much for making the PR by the way!)

Awesome! I'll test the 3.0.0 version asap, to confirm that it does fix my issue!
However, I still do think that this test should be merged (I changed the base branch to master), since I would consider this behaviour as part of the "public API", and breaking this test could break things in some apps :)

Ah right, sorry I misunderstood the PR slightly. I'll go ahead and merge.
Thank you!

source: gharchive/pull-request

authors: AugustinLF codecov-io coveralls ellbee

repo: reactjs/reselect


text:
Fix typos in hooks-overview

@LFedorov спасибо!


text:
🛑 Bellshade Website is down

In 7a9e080, Bellshade Website (https://bellshade.org) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Bellshade Website is back up in c47a946 after 5 minutes.


text:
WebPack config error? css-hot-loader used in production mode!

https://www.npmjs.com/package/css-hot-loader
https://github.com/readium/readium-desktop/blob/541e48bbe45407a8918018e0076d4ca5ebecd68b/webpack.config.renderer.js#L188
https://github.com/readium/readium-desktop/blob/541e48bbe45407a8918018e0076d4ca5ebecd68b/webpack.config.reader.js#L175

I think the culprit is this commit from April 1st (April's fool? ;)
https://github.com/readium/readium-desktop/commit/1b98c500f427f7f1ea07b89917d6c2486aa656ef
It looks like a "typo" introduced by mistake in the condition checking for production vs. development mode.
I suggest we move css-hot-loader from prod to dev build configuration.

repo: readium/readium-desktop


text:
Customization options for bitbucket, gitlab, and github are undocumented

I had to search the source-code to learn of the existence of these options for 'edit page' links.  These should be documented properly with descriptions of their usages.

Yeah I need to write some documentation.

Based on https://github.com/readthedocs/sphinx_rtd_theme/issues/231#issuecomment-126447493 from @IKavanagh (thank you!), you can do the following in your conf.py file:
html_context = {
  "display_github": True, # Add 'Edit on Github' link instead of 'View page source'
  "github_user": "peterjc",
  "github_repo": project,  # assuming an exact match
  "github_version": "master",
  "conf_py_path": "/doc/", # needs leading and trailing slashes!
  # "source_suffix": source_suffix,
}

I've not explored why conf_py_path is called that, but it seems to be the relative path from the GitHub repository root to your documentation folder (where conf.py lives etc), and it needs leading and trailing slashes.
I have not needed to set source_suffix myself yet.
e.g. https://github.com/peterjc/thapbi-pict/commit/9ec5300c8bf788a693fa6f893a1494170e14bd0f
If you need to over-ride this in any particular *.rst files, for example automatically generated ones which are not in GitHub, they need a line like the following added at the start:
:github_url: https://github.com/peterjc/...

e.g. https://github.com/peterjc/thapbi-pict/commit/f14b1df17ed3faf382a63095aa6e22f519e71957 does this via a function in conf.py

@cglacet you can customize that with the vcs_pageview_mode option  (defaults to blob for gitlab, but currently is broken https://github.com/readthedocs/sphinx_rtd_theme/pull/1010

What would be the config for GitLab users?

While on the subject, the URL for bitbucket is hardcoded to be "bitbucket.org"
https://github.com/readthedocs/sphinx_rtd_theme/blob/0da22b885be387b79f7552c92be00fd14d11228a/sphinx_rtd_theme/breadcrumbs.html#L41-L47

authors: Blendify jpsenior jreed1701 peterjc stsewd thompol


text:
Ability to delete files

Thanks in advance guys!
So pull, publish works great but is it possible to delete files straight from Code?

@sg-chrishasz, as a proposal for this feature: sppurge can be integrated in the same way like sppull and spsave, it used the same principals on the background.

@quachhongak, I referenced to a library which could be integrated to SPGo by the extension authors. It's a raw Node.js library and some reasonable affords needed to add it to VS Code extension. Yet, as a console command, it can be easily executed i.e. as a Gulp task.

@koltyakov I will look into sppurge as a solution for file deletes. Thanks for the reference!

I have just published 1.3.0 to the VS Marketplace which supports file delete as a context-menu command. Thanks for the feedback and support of SPGo! Closing this issue.

Thank YOU!!!

authors: koltyakov quachhongak sg-chrishasz

repo: readysitego/spgo


text:
Possible memory visibility issue in  PublicationImage

We believe we found some misbehavior with respect to the memory visibility of the buffer that Aeron returns while polling a subscription. We are also linking a barebones setup that is able to reproduce the misbehavior, can be found here.
https://github.com/dkonik/AeronTesting
The Behavior:
We observe that the bytes inside the buffer that is passed to the FragmentHandler while polling seem to (on occasion) change inside the handler. More specifically, at the beginning of the FragmentHandler we call buffer.getLong(offset), then we do some work, then at the end of the FragmentHandler we call buffer.getLong() again and the value that is returned is not the same as the first time getLong was called. For example, with the following code in the FragmentHandler:
final long receivedIndex = buffer.getLong(offset);

if (receivedIndex != previous + 1) {
	System.out.println("MISSED MESSAGE AT INDEX: " + previous + "->" + receivedIndex);
	System.out.printf("%x %x %x %x %x %x %x %x\n", buffer.getByte(offset), buffer.getByte(offset + 1),
			buffer.getByte(offset + 2), buffer.getByte(offset + 3), buffer.getByte(offset + 4),
			buffer.getByte(offset + 5), buffer.getByte(offset + 6), buffer.getByte(offset + 7),
			buffer.getByte(offset + 8));
	System.out.println("Offset(again): " + buffer.getLong(offset) + "\n");
}

++messageNum;
previous = receivedIndex;

We get output of:
MISSED MESSAGE AT INDEX: 188610->96091
c3 e0 2 0 0 0 0 0

Offset(again): 188611

MISSED MESSAGE AT INDEX: 96091->188612

c4 e0 2 0 0 0 0 0

Offset(again): 188612

MISSED MESSAGE AT INDEX: 371101->278582

9e a9 5 0 0 0 0 0

Offset(again): 371102

MISSED MESSAGE AT INDEX: 278582->371103

9f a9 5 0 0 0 0 0

Offset(again): 371103

As you can see, for the first one, you would expect the returned value to be 188610 + 1, but the first time getLong is called a different number is returned (96091). Afterwards, when getLong is called again, the correct value is returned, implying that Aeron is giving us a buffer before the memory contained by that buffer is fully visible in all threads.
Additionally, we have only observed this while sending 5-10 packets back to back as fast as possible (as opposed to say, a one packet at a time ping-pong style).
System Configurations:
Both boxes are Haswells with a direct connection between them (no switch), and rhel 7 version 3.10.0-327.22.2.el7.x86_64

We are using the Oracle JRE 1.8.0_92-x86_64 (bug also manifests itself on the JDK of the same verison) and we’re building the latest HEAD of Aeron (25ab0b382c4784636929ce032c897d2c41e23562).

Hi guys,
Only as hint...
The producer in the test put the new values using:
atomicBuffer.putLong(0, i);
or inlined:
UNSAFE.putLong(byteArray, addressOffset + index, bits);
At the end of the producer loop there is no esplicit release of the unsafe buffer, hence no direct usage of the byteBuffer reference (nor for reading/wrinting) between the instantiation and the end of the call.
I do not see (and i can be wrong!) any good reason why the compiler has to retain that (direct) ByteBuffer and probably there are chances it will be released at some point.
Maybe the issue is not on the consumer side...but on the producer...

@franz1981 I don't get your point on the producer side. The code looks fine to me. The direct buffer is referenced from within the atomicBuffer and it is used repeatedly within the test.
What do you mean by "no esplicit release of the unsafe buffer"?

@mjpt777 Sorry, i need to explain it better.
About the explicit release of the unsafeBuffer I mean a call on Cleaner::clean against the direct buffer referenced into the unsafeBuffer.
AFAIK any usage of the byteBuffer field (including the mentioned "release" of it) after the loop is fine to tell the compiler to not collect it between all the puts and copyMemory inside the producer loop, because even if the unsafeBuffer instance is used,its byteBuffer field doesn't seem to have other uses after its instantiationn.
I'm reading all the source code with a smart phone and maybe I've missed other usages of that field or misunderstood how the liveness analysis of the compiler work for unused instance fields...

I've been able to recreate this once in an independent test. I have found a few issues in the referenced test. For example there are 9 bytes, and not 8, read in the printf output. Also System.exit is not a great way to shut down such a system. These issues should be independent.

@franz1981 I can make this test fail with the srcBuffer stored in a field of the class. See the commit https://github.com/real-logic/Aeron/commit/e22c98935ab94d892ffc3aad7bbb455d9442a9ee .

I've been playing with the test and if I create messages that align on rotation, i.e. 96 byte message length, then the bytes observed are the value in the term buffer from 3 terms previous. This suggests we have a problem with cleaning and observing the previous value within a message. Term length is 16MB.
Issue at message number transition: 534670 -> 141455
offset: 1329056
length: 96
expected bytes: 0 0 0 0 0 2 28 8f
received bytes: 0 0 0 0 0 0 28 8f
messageValue on second read: 534671
messageValue on third read: 534671

@mjpt777 Thank you Martin to have used your time to test it!
It is so aleatory to be sure at 100% and is difficult to say how long these "tricks" could work with a very smart compiler
They belong to the same family of the "padding tricks" :(

@franz1981 To be clear the issue does not seem to be what you have suggested. I don't see that as the problem. This is not a "trick".

For another data point, I cannot make the test fail on Windows 10.

@mjpt777 I apologize, I have to be more precise with the terms! :)

And another data point, I can't make the system test fail on Mac.

Also it does not seem to manifest if the driver is run in SHARED or SHARED_NETWORK threading mode which might mean a resource starvation issue.

I don't think it is a partial write problem given this failure.
The difference between the expected message and the first read is exactly 3 terms worth of messages.
Three terms:
1024 * 1024 * 16 * 3  = 50,331,648
Message number difference:
736981 - 343765 = 393216
393216 * 128 = 50,331,648
Issue at message number transition: 736980 -> 343765
offset: 10447520
length: 96
expected bytes: 0 0 0 0 0 2 3e d5
received bytes:  0 0 0 0 0 1 3e d5
expected bits: 10110011111011010101
received bits:    1010011111011010101
messageValue on second read: 736981
messageValue on third read: 736981

Had another failure with exactly 3 terms difference again.
Issue at message number transition: 787265 -> 394050
offset: 106784
length: 96
expected bytes: 0 0 0 0 0 3 3 42
received bytes: 0 0 0 0 0 1 3 42
expected bits: 11000000001101000010
received bits: 1100000001101000010
messageValue on second read: 787266
messageValue on third read: 787266

Same here.

The last time the buffer was read is the value there. So, it definitely looks like a stale value. Although, you would think it would have been cleaned.

If I rewrite the TermRebuilder code to do the following it still fails which further suggests it is the zeroing out of the memory which is the culprit or the getLong returning a stale value.
    public static void insert(final UnsafeBuffer termBuffer, final int termOffset, final UnsafeBuffer packet, final int length)
    {
        /*
        final int firstFrameLength = packet.getInt(0, LITTLE_ENDIAN);
        packet.putInt(0, 0);
        UnsafeAccess.UNSAFE.storeFence();

        termBuffer.putBytes(termOffset, packet, 0, length);
        frameLengthOrdered(termBuffer, termOffset, firstFrameLength);
        */

        for (int i = 8; i < length; i += 8)
        {
            termBuffer.putLong(termOffset + i, packet.getLong(i));
        }

        termBuffer.putLongOrdered(termOffset, packet.getLong(0));
    }

Issue at message number transition: 560123 -> 166908
offset: 4587040
length: 96
expected bytes: 0 0 0 0 0 2 8b fc
received bytes: 0 0 0 0 0 0 8b fc
expected bits: 10001000101111111100
received bits: 101000101111111100
messageValue on second read: 560124
messageValue on third read: 560124

@kdkavanagh With the latest change and release, this is worth retesting.

Thanks, will give it a try in the next few days and report back here

Apologies for the delayed feedback - Looks good.  Thanks guys.

@kdkavanagh Thanks for re-testing.

repo: real-logic/Aeron


text:
Possible bug in ManyToOneRingBuffer#unblock, better unit test

Ok, sorry for the previous issue #159,
So basically, the issue is in my opinion, that the unblock does not work as it should when the ring buffer is full, here is a unit test which can be added to the org.agrona.concurrent.ringbuffer.ManyToOneRingBufferTest. This test should hopefully reproduce the issue.
    public void shouldUnblockGapWithZerosWhenRingBufferIsFull()
    {
        final int messageLength = ALIGNMENT * 4;
        when(buffer.getLongVolatile(HEAD_COUNTER_INDEX)).thenReturn((long)messageLength);
        when(buffer.getLongVolatile(TAIL_COUNTER_INDEX)).thenReturn((long)messageLength + CAPACITY);

        when(buffer.getIntVolatile(messageLength * 2)).thenReturn(messageLength);

        assertTrue(ringBuffer.unblock());

        final InOrder inOrder = inOrder(buffer);
        inOrder.verify(buffer).putInt(typeOffset(messageLength), PADDING_MSG_TYPE_ID);
        inOrder.verify(buffer).putIntOrdered(lengthOffset(messageLength), messageLength);
    }

I just took the unit test org.agrona.concurrent.ringbuffer.ManyToOneRingBufferTest#shouldUnblockGapWithZeros and set the mock so it returns on getLongVolatile(TAIL_COUNTER_INDEX) messageLength + CAPACITY which should mean that the ring buffer is full.
I hope this unit test is clearer, than the one from previous issue.
Just to elaborate on the issue, we use the ManyToOneRingBuffer, and its underlying buffer is mmap from a file. What we experienced is that the underlying file get corrupted and at the same time the ring buffer was full. I restarted our system, but the buffer was block on the next start and invoking the unblock method didn't help.

Added some tests and a fix. Thanks this is a better report.

Thanks for quickly fixing this. I will close the issue.

repo: real-logic/agrona


text:
Storing an array of strings using Realm's RLMArray [Objective-C]

Does anyone know how you can use Realm to store an array of strings? I'm trying to map the following response into Realm correctly:
"zoneInfo": {
    "tariffInfoLines": [
        "In this city you pay per minute."
    ]
}

We have a zoneInfo object that contains a tariffInfoLines array. This tariffInfoLines array contains strings. In Realm there are two different variable types for storing data. The first is RLMObject which allows your standard NSString, int, long etc.
The second type is RLMArray, which is used for arrays (as NSArray is not supported). You have to give the array a type, which must be a class that subclasses RLMObject. We have so far got around this by using a ABCRealmString object, as shown below:
@property RLMArray<ABCRealmString> *tariffInfoLines;

ABCRealmString contains an NSString property (it is basically a wrapper):
@property NSString *value;

However what this means is that when Realm tries to map the response to persist the data, it is looking for a value for the key "value" (the name of the property). It appears that it expects a response similar to the following:
"zoneInfo": {
    "tariffInfoLines": [
    	{
    		"value": "In this city you pay per minute."
    	},
    ]
}

In the project, we have it working for the following structure:
"userOptions": [
	{
		"wantsEmailNotifications": true,
		"wantsPushNotifications": false
	},
]

This has an array, with objects inside that have clear key value pairs that Realm can map to. The zoneInfo structure appears to be the only place that we have an array with sets of values inside without them being inside an object or having any keys.
If anyone could shed some light on this, regarding if this is possible using Realm, or whether an API change is required to match a structure that Realm can map.

Although this example demonstrates how to store flat arrays of strings on a Realm model, you can extend this pattern to store anything from arrays of integers to native Swift enum's. Basically anything that you can map to a representable type in Realm.
class RealmString : Object {
    dynamic var stringValue = ""
}

class Person : Object {
    var nicknames: [String] {
        get {
            return _backingNickNames.map { $0.stringValue }
        }
        set {
            _backingNickNames = newValue.map { RealmString(value: [$0]) }
        }
    }
    dynamic var _backingNickNames = List<RealmString>()
}

Thanks for your response @jpsim. I'm looking to persist data using realm with an array that has no keys. Please see an example below:
"tariffInfoLines": [
    "In this city you pay per minute."
]

Using the solution outlined above, this results in the following error:
Invalid value 'In this city you pay per minute.' to initialize object of type 'ABCRealmString': missing key 'value'
Call stack
0   __exceptionPreprocess
1   objc_exception_throw
2   RLMValidatedValueForProperty
3   RLMCreateObjectInRealmWithValue

RLMValidatedValueForProperty is available in RLMObjectBase.mm on Line 334
id RLMValidatedValueForProperty(id object, NSString *key, NSString *className) {
    @try {
        return [object valueForKey:key];
    }
    @catch (NSException *e) {
        if ([e.name isEqualToString:NSUndefinedKeyException]) {
            @throw RLMException([NSString stringWithFormat:@"Invalid value '%@' to initialize object of type '%@': missing key '%@'",
                                 object, className, key]);
        }
        @throw;
    }
}

Parameters passed in:
object = "In this city you pay per minute." [NSString]
key = "value" [NSString]
className = "ABCRealmString" [NSString]

ABCRealmString:
@interface PMGRealmString : RLMObject
    @property NSString *value;
@end

If this returned object when there are no keys, then it would allow a flat structure (as previously mentioned) to be used with Realm.

Here's what the "wrapper property" approach looks like in Objective-C, along with a demonstration of setting and reading a flat array of strings to the realm object:
// Models
@interface RealmString : RLMObject
@property NSString *stringValue;
@end
@implementation RealmString
@end

RLM_ARRAY_TYPE(RealmString)

@interface TariffInfo : RLMObject
@property RLMArray<RealmString> *storedTariffInfoLines;
@property NSArray *tariffInfoLines;
@end
@implementation TariffInfo
+ (nullable NSArray *)ignoredProperties {
    return @[@"tariffInfoLines"];
}

- (void)setTariffInfoLines:(NSArray *)tariffInfoLines {
    [self.storedTariffInfoLines removeAllObjects];
    for (NSString *line in tariffInfoLines) {
        [self.storedTariffInfoLines addObject:[[RealmString alloc] initWithValue:@[line]]];
    }
}

- (NSArray *)tariffInfoLines {
    return [self.storedTariffInfoLines valueForKey:@"stringValue"];
}
@end

// Usage
RLMRealm *realm = [RLMRealm defaultRealm];
[realm transactionWithBlock:^{
    TariffInfo *info = [[TariffInfo alloc] init];
    // Set an array of strings
    info.tariffInfoLines = @[@"In this city you pay per minute."];
    [realm addObject:info];
}];
NSLog(@"Tariff Info: %@", [TariffInfo allObjects]);
// Prints
//
// Tariff Info: RLMResults <0x7fa7b0727310> (
//  [0] TariffInfo {
//      storedTariffInfoLines = RLMArray <0x7fa7b04ab760> (
//          [0] RealmString {
//              stringValue = In this city you pay per minute.;
//          }
//      );
//  }
// )
NSLog(@"Tariff Info Lines: %@", [[[TariffInfo allObjects] firstObject] tariffInfoLines]);
// Prints
//
// Tariff Info Lines: (
//  "In this city you pay per minute."
// )

I hope this helps!

Thanks for your help @jpsim. That has fixed my issue!

Glad to hear it!

i cannot seem to achieve this in Swift. I need List of Int type.
class OfficeTimingData:Object, Mappable{
dynamic var fromHour:String?
dynamic var toHour:String?
var days:[Int] // Not Working!!!!

required convenience init?(_ map: Map) {
    self.init()
}

func mapping(map: Map) {
    fromHour <- map["fromHour"]
    toHour <- map["toHour"]
    days <- map["days"]
}

}

@jpsim is this technique still relevant in 0.97.0 ?
dynamic var _backingNotes = List<RealmString>() generates this compilation error msg :
Property cannot be marked dynamic because its type cannot be represented in Objective-C

@jpsim  Thank you for your example.

@jpsim The first Swift example you wrote produces a compiler error with this message:
map produces [T], not the expected contextual result type List<RealmString>
However this seems, to go  away if I forcibly cast it, e.g. as! List<RealmString>

Here's a revised version:
class RealmString: Object {
    dynamic var stringValue = ""
}

class Person: Object {
    var nicknames: [String] {
        get {
            return _backingNickNames.map { $0.stringValue }
        }
        set {
            _backingNickNames.removeAll()
            _backingNickNames.appendContentsOf(newValue.map { RealmString(value: [$0]) })
        }
    }
    let _backingNickNames = List<RealmString>()

    override class func ignoredProperties() -> [String] {
        return ["nicknames"]
    }
}

Usage:
let realm = try! Realm()
try! realm.write {
    let person = Person()
    person.nicknames = ["Doug", "Dougie"]
    realm.add(person)
}

print(realm.objects(Person.self).first!.nicknames)
// Prints: ["Doug", "Dougie"]

Hello @jpsim
I made as you describe , but I get error "'RLMException', reason: 'Property 'confirmedMembersIds' is declared as 'NSArray', which is not a supported RLMObject property type. ", how can I fix it?
class RealmString: Object {

    dynamic var stringValue = ""

}

class CardModel: Object, Mappable {
 override static func ignoredProperties() -> [String] {
        return ["confirmedMembersIds, pendingRequests"]
    }

var confirmedMembersIds: [String] {
        get {
            return _confirmedMembersIds.map { $0.stringValue }
        } set {
            _confirmedMembersIds.removeAll()
            _confirmedMembersIds.append(objectsIn: newValue.map { RealmString(value: [$0]) })
        }
    }
    private let _confirmedMembersIds = List<RealmString>()

    var pendingRequests: [String] {
        get {
            return _pendingRequests.map { $0.stringValue }
        } set {
            _pendingRequests.removeAll()
            _pendingRequests.append(objectsIn: newValue.map({ RealmString(value: [$0]) }))
        }
    }
    private let _pendingRequests = List<RealmString>()
}

override static func ignoredProperties() -> [String] {
    return ["confirmedMembersIds, pendingRequests"]
}

You're telling Realm to ignore a single property named confirmedMembersIds, pendingRequests, which isn't what you want. Your implementation should instead be:
return ["confirmedMembersIds", "pendingRequests"]

@bdash
Thank you very much

authors: JamieCruwys NikKovV ahmedk92 alexsanderkhitev bdash guiracine jpsim sudeep23

repo: realm/realm-cocoa


text:
Saving a realm to-many relationship

I'm having some problems with realm and relationships.
Here's my models:
class CreditCard: Object {
    dynamic var owner: String?
    var purchases: List<Purchase>()
    ...
}

And the second class which looks like that:
class Purchase: Object {
    dynamic var id: Int = 0
    dynamic var name: String?
    dynamic var data: String?
    let creditCard = LinkingObjects(fromType: CreditCard.self, property: "purchases")
    ...
}

All information comes from a web server. When the app opens, the first view controller makes a network request and download some credit card information. I can save all CreditCard objects just fine:
try! realm.write {
    for creditCard in creditCards { // creditCards = all dictionary from my json object
        let newCreditCard = CreditCard()
        newCreditCard.id = creditCard["Id"].intValue
        ...
        realm.add(newCreditCard, update: true)
    }
}

On a second view controller, I have another request that downloads all purchases made by a credit card. After the request this is how I try to write to realm:
let creditCard = realm.objects(CreditCard.self).first // there's some code to get this.
try! realm.write {
    for purchase in purchases { // purchases = objects from a son
        let newPurchase = Purchase()
        newPurchase.id = purchase["Id"].intValue
        newPurchase.name = purchase["name"].stringValue
        newPurchase.date = purchase["date"].dateValue
        realm.add(newPurchase, update: true)
        creditCard.purchases.append(newPurchase)
    }
 }

Realm Browser shows that all credit card and purchases are saved. The problem happens when I open the app again, and try to create/update all credit cards again (inside the first view controller). For some reason, this is breaking the relationship between the credit card and its purchases.
The purchases are saved on my realm objects, but the relationship between CreditCard and Purchase is lost. So, the return from the code below is always empty when I re-open the app:
let purchases = creditCard.purchases

Does realm.add(newCreditCard, update: true) overwrite all purchases? What am I doing wrong?
Realm version: 2.0.3
Xcode version: 8.1
iOS/OSX version: 10

Does realm.add(newCreditCard, update: true) overwrite all purchases? What am I doing wrong?

When update: true is passed to Realm.add(_:update:), Realm looks for an existing object with the given primary key. If it finds one, it updates all of the object's properties to the values specified in the passed-in object. This means that the purchases property of the credit card object will be updated to contain only the objects in newCreditCard.purchases. You don't appear to populate this field in the code snippet you've provided, so it's quite likely that you're unintentionally clearing the field by doing this.

Thanks for the clarification. You're right. I don't populate newCreditCard.purchases when my request finishes. I misunderstood how update:true works. I was thinking that it would update all fields that changed, but keep the existing ones as before. I would only update the limit, for example, but keep all purchases, name and so on.
What would be the best approach to keep all credit card purchases when updating it? Should I write a query to verify if the credit card already exists and populate the newCreditCard.purchases with he result? Is there a better way to do this?

My suggestion would be to use Realm.object(ofType:forPrimaryKey:) to retrieve the existing object based on its primary key, then set its properties as you do now. If object(ofType:forPrimaryKey:) returns nil, instantiate a new object and set its properties instead. It'd look something like:
try! realm.write {
    for creditCard in creditCards { // creditCards = all dictionary from my json object
        let id = creditCard["Id"].intValue
        let creditCard = realm.object(ofType: CreditCard.self, forPrimaryKey: id) ?? CreditCard(value: ["id": id])
        ...
        realm.add(newCreditCard)
    }
}

This relies on the fact that Realm.add(_:) is a no-op if the object to be added already exists in the Realm.

Perfectly. Worked like a charm. Thank you for your time and attention.

authors: bdash hdoria


text:
We get a probable for help. The crash report get information "Too many open files"

/var/mobile/Containers/Data/Application/E9EE7F3D-3BAA-42A7-A2AD-E9D4DC8A9D7F/Documents/ShopOrderTab.realm.management/log_b: Unable to open a realm at path '/var/mobile/Containers/Data/Application/E9EE7F3D-3BAA-42A7-A2AD-E9D4DC8A9D7F/Documents/ShopOrderTab.realm.management/log_b': open() failed: Too many open files.
(null)
Realm framework version: ?
1.1.0
Xcode version: ?
8.3.3

Please check out https://github.com/realm/realm-cocoa/issues/4435 and provide some more info about your case in order to reproduce and investigate it, thanks!

authors: betterton stel


text:
Should constantly try to reconnect regardless of ReachabilityStatus

Because Reachability doesn't guarantee that a packet can actually reach the target host. It is about the local network configuration (interface up / down) and if it can send a packet.

A remote host is considered reachable when a data packet, sent by an application into the network stack, can leave the local device. Reachability does not guarantee that the data packet will actually be received by the host.

SCNetworkReachability | Apple Developer Documentation

Important: The SCNetworkReachability API is not intended for use as a preflight mechanism for determining network connectivity. You determine network connectivity by attempting to connect. If the connection fails, consult the SCNetworkReachability API to help diagnose the cause of the failure.

Design for Variable Network Interface Availability
There are situations where we can not connect to ROS even if the device's network connection is fine. (e.g. ROS is down, 100% packet loss by Network Link Conditioner) In those cases, it will not reconnect forever since ReachabilityStatus does not change after failing connection once.
Goals
Sync will resume when the server recovers even if the app launched while the server is down.
Expected Results
An app tries to reconnect to ROS constantly.
Actual Results
If an app fails first (ROS is down or 100% packet loss), never reconnect again.
Steps to Reproduce

Launch sync example apps without ROS
Launch ROS after the app launched
Sync never happen

Version of Realm and Tooling
Realm framework version: 2.8.3
Realm Object Server version: 1.8
Xcode version: 8.3
iOS/OSX version: 1.3
CC @austinzheng @jpsim @bdash

As far as I'm aware, we'll always try to reconnect so long as we have a sync session active. It's just that we'll increase the interval between reconnection attempts when they fail. We use reachability changes as a trigger to reset the interval between reconnection attempts. When we fail to reconnect in this case, is it simply because the reconnect interval has grown very large? Or is something else at play?

I agree with Mark. I'll have to check again, but as far as I remember reachability is only used to speed up reconnection, never to delay or stop it.

My explanation may not be accurate. Never happening again is refreshing server token.
If an app launched while ROS is down, refreshing token failed then try to schedule refreshing 10 seconds later,
https://github.com/realm/realm-cocoa/blob/e44a4760337c2c70d99e54aff74047ac30540ee8/Realm/RLMSyncSessionRefreshHandle.mm#L184-L209
But it is not scheduled.
https://github.com/realm/realm-cocoa/blob/e44a4760337c2c70d99e54aff74047ac30540ee8/Realm/RLMSyncSessionRefreshHandle.mm#L93-L113
As a result, sync will not happen unless ReachabilityStatus changes.

We only retry the token refresh for a certain subset of errors. Maybe we should just retry forever for any error. https://github.com/realm/realm-cocoa/blob/e44a4760337c2c70d99e54aff74047ac30540ee8/Realm/RLMSyncSessionRefreshHandle.mm#L184-L204

I think the problem is fireDateForTokenExpirationDate returns nil.
https://github.com/realm/realm-cocoa/blob/e44a4760337c2c70d99e54aff74047ac30540ee8/Realm/RLMSyncSessionRefreshHandle.mm#L93-L98
Is the behavior of that as intended?

authors: austinzheng bdash kishikawakatsumi


text:
Typo in Realm.swift

Returns whether two Realm isntances are equal.

isntances -> instances
see https://github.com/realm/realm-cocoa/blob/2493b356b3422f3bbb4dab27dc351f16ebdda13c/RealmSwift/Realm.swift#L675

Fixed in 339605491.

authors: jpsim torstenlehmann


text:
attempt to fix issues with launching simulators from different Xcodes

I ran this successfully a few times on Test Xcode 6 Installation Examples and Test Xcode 7 Installation Examples. /cc @segiddins
Definitely hacky, but this is just as part of our tests so my standards are pretty low especially since things are so broken currently.

:+1: assuming the tests don't fail due to hung simulators :)

due to hung simulators :)

with our luck...

Only failure was #2294 again.

authors: jpsim segiddins


text:
Allow Customer creation without card but with trial_end

I've tested it with live: true flag. It passes.
However I get 10 failures :
51 examples, 10 failures, 1 pending

Failed examples:

rspec ./spec/shared_stripe_examples/validation_examples.rb:16 # StripeMock::Instance behaves like Card API retrieval and deletion with recipients runs a luhn check for charges
rspec ./spec/shared_stripe_examples/card_examples.rb:182 # StripeMock::Instance behaves like Card API retrieval and deletion with recipients can retrieve all recipient's cards
rspec ./spec/shared_stripe_examples/card_examples.rb:187 # StripeMock::Instance behaves like Card API retrieval and deletion with recipients deletes a recipient card
rspec ./spec/shared_stripe_examples/card_examples.rb:193 # StripeMock::Instance behaves like Card API retrieval and deletion with recipients deletes a recipient card then set the default_card to nil
rspec ./spec/shared_stripe_examples/card_examples.rb:203 # StripeMock::Instance behaves like Card API retrieval and deletion with recipients deletion when the recipient has two cards has just one card anymore
rspec ./spec/shared_stripe_examples/card_examples.rb:210 # StripeMock::Instance behaves like Card API retrieval and deletion with recipients deletion when the recipient has two cards sets the default_card id to the last card remaining id
rspec ./spec/shared_stripe_examples/charge_examples.rb:227 # StripeMock::Instance behaves like Charge API when use starting_after param
rspec ./spec/shared_stripe_examples/customer_examples.rb:41 # StripeMock::Instance behaves like Customer API creates a stripe customer with a dictionary of card values
rspec ./spec/shared_stripe_examples/recipient_examples.rb:97 # StripeMock::Instance behaves like Recipient API Errors throws an error when the customer does not have the retrieving card id
rspec ./spec/shared_stripe_examples/transfer_examples.rb:67 # StripeMock::Instance behaves like Transfer API when amount is not integer

This is not this PR scope but maybe useful.

@antoinelyset good job! thank you. One thing: can you add live: true for 'creates a customer when trial_end is set and no source' test and commit it?

Of course !
Just to know what is the politic for this flag ?

as I know: "all tests should be tested in live mode except tests when some items should be stored locally". I see your error. Look at  strategies for tests

lgtm :)

authors: alexmamonchik antoinelyset

repo: rebelidealist/stripe-ruby-mock


text:
Dark theme button is not working

I would like to fix this bug please assign this issue to me

@sanjay-kv please assign this issue to me

have to put on hold, i can see its working for me.

authors: Ananya-vastare sanjay-kv

repo: recodehive/awesome-github-profiles


text:
💡[Feature]: AI-Powered Super-Resolution and Image Restoration for Historical Photos

Is there an existing issue for this?

[X] I have searched the existing issues

Feature Description
This project aims to create an AI-Powered Image Restoration Tool that transforms low-quality historical photos by upscaling, denoising, and adding realistic color. Using deep learning, it revives old images with stunning clarity and detail, preserving memories in a vibrant, lifelike form. Perfect for heritage conservation and digital archiving!
Use Case
This model will help to restore Old photos by upscaling it or adding realistic color etc.
Benefits
No response
Add ScreenShots
No response
Priority
High
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

@sanjay-kv
Kindly assign me the issue

repo: recodehive/machine-learning-repos


text:
[FEATURE] Integrate services in Import Subdomains features, Root Domain details page

Integrate services in Import Subdomains features, Root Domain details page
Mockups

@YanetABUC this is going to be a list of subdomain in a .txt
Ex
mysub.yahoo.com
other.yahoo.com
other2.yahoo.com

POST api/rootdomains/uploadSubdomains/{targetName}/{rootDomainName}

authors: YanetABUC gorums

repo: reconness/reconness-frontend


text:
[ArgumentValueResolverToValueResolverRector] Add rule (#289)

Fixes #289

Don't forget to update config/sets/symfony/symfony62.php by adding this rule and remove the RenameClassRector lines.

@samsonasik its ready to be reviewed now.

@samsonasik can you have another look when you have time?

btw, @JohJohan @johanadivare are you same person 😄 ?

btw, @JohJohan @johanadivare are you same person 😄 ?

Haha yes, @johanadivare is my work Github ;p

@samsonasik i moved the files to the correct folders

Thank you @johanadivare , @TomasVotruba lets merge it ;)

Thanks a lot @johanadivare :+1:

authors: JohJohan TomasVotruba johanadivare samsonasik stefantalen

repo: rectorphp/rector-symfony


text:
Update owners to have a dspo rep as approver.

Description
It probably makes sense to have one of the dspo component owners to be able to dev-approve prs pertaining to DSPO/DSP.
cc @accorvin
How Has This Been Tested?
No code change.
Merge criteria:

[x] The commits are squashed in a cohesive manner and have meaningful messages.

The following does not apply as this is not a code change.

[ ] For commits that came from upstream, [UPSTREAM] has been prepended to the commit message.
[ ] JIRA link(s):
[ ] The Jira story is acked.
[ ] Live build image:
[ ] Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
[ ] The developer has manually tested the changes and verified that the changes work.
[ ] QE contact acknowledges that this has been tested and is approved for merge.

done

repo: red-hat-data-services/odh-deployer


text:
🧹hack scripts shebang cleanup

Hack scripts shebang cleanup
Improve portability among platforms. Hack script code depends on bash version 5 syntax features. Replace '#!/bin/bash' with  '#!/usr/bin/env bash' as a shebang for hack scripts.
Specifically enable 'make gen-latest-csv' on MacOS:
➜  hack git:(hack_cleanup) ✗ /bin/bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin21) <--- insufficient version 3!
Copyright (C) 2007 Free Software Foundation, Inc.
➜  hack git:(hack_cleanup) ✗ /usr/local/bin/bash --version
GNU bash, version 5.1.16(1)-release (x86_64-apple-darwin21.1.0)
Copyright (C) 2020 Free Software Foundation, Inc.
Signed-off-by: Alexander Indenbaum aindenba@redhat.com

/ok-to-test

repo: red-hat-storage/ocs-operator


text:
Fix race condition in Cassandra span reporting

The Cassandra driver is asynchronous under the hood. It runs IO in a reactor on another thread (or in the native reactor if already async, e.g. gevent). It provides an API that returns Futures. You can either attach callbacks to the future to get notified of completion/failure or call result() on the future to block the current thread until the result comes in. Callbacks are called on the IO thread or in the registering thread if the future had already completed at time of registration. The result() API works by blocking on a threading.Event object. When the operation finishes inside the IO thread, the future signals the Event and then calls each of the callbacks in turn in that order.
Baseplate's Cassandra instrumentation intercepts Futures and attaches callbacks to them before returning them to the application. This is how we mark spans finished. The execute_async() API works this way by default and the execute() API is just execute_async() with an immediate call to result() on the returned future.
Because the IO thread signals the event which unleashes the main thread's result() before doing the callbacks, there's a race condition where the main thread finishes up its work and finishes the server span before the callbacks and therefor the child span get to finish.
The impact of this is that a child span might finish after its parent span sometimes. For most observers this should be fine. The metrics observer will lose that child span's metric though because the batch for the server span will have already shipped. This means undercounting Cassandra calls and timings.

I opened a speculative PR (see above) against the upstream driver and got some feedback/suggestions. Not sure the best way to fix this right now. It's also causing us issues on r2 when we have the worker threads try to send metrics.

Fixed by the venerable @bsimpson63 in d366da31a95b69e2509f24c5b8538630ce0d6923


text:
Add baseplate.lock

Upstreaming from thing service and extended to support redis.
👓 @spladug @cshoe @pacejackson

Mostly wondering if we want to promote doing the old-style janky memcached locks in future applications.

Yeah, 🤷♀  . In addition to the problems with our standard lock implementation, we might not want to encourage heavy use of locks because they should often be a last resort. I guess if a service really really needs locks they can implement it themselves and it'll be more of a "proceed at your own peril" situation rather than being blessed by the framework.
If there were a way to make an interface that could eventually be eventually switched to use redlock that could be worth exploring.
I don't fee strongly about this PR or any of the options for including locks (or not) in baseplate.

🏃

repo: reddit/baseplate


text:
Updating top-level .gitignore to ignore .orig files

Updating .gitignore to ignore detritus from merge conflicts.

:+1:

:+1:

authors: doug-wade gigabo roblg

repo: redfin/react-server


text:
Appstudio update test-component-pac-aray

Pipelines as Code configuration proposal
To start the PipelineRun, add a new comment with content /ok-to-test
For more detailed information about running a PipelineRun, please refer to Pipelines as Code documentation Running the PipelineRun
To customize the proposed PipelineRuns after merge, please refer to Build Pipeline customization

Pipelines as Code CI/test-component-pac-aray-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
6 seconds
init

✅ Succeeded
15 seconds
clone-repository

✅ Succeeded
2 minutes
sast-snyk-check

✅ Succeeded
1 minute
build-container

✅ Succeeded
13 seconds
inspect-image

✅ Succeeded
14 seconds
deprecated-base-image-check

✅ Succeeded
1 minute
clamav-scan

✅ Succeeded
32 seconds
clair-scan

✅ Succeeded
8 seconds
sbom-json-check

✅ Succeeded
13 seconds
show-sbom

✅ Succeeded
12 seconds
show-summary


text:
Appstudio update test-component-pac-wueg

Pipelines as Code configuration proposal

Pipelines as Code CI/test-component-pac-wueg-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
19 seconds
init

✅ Succeeded
21 seconds
clone-repository

✅ Succeeded
1 minute
build-container

✅ Succeeded
18 seconds
sanity-inspect-image

✅ Succeeded
18 seconds
deprecated-base-image-check

✅ Succeeded
49 seconds
clamav-scan

✅ Succeeded
3 minutes
clair-scan

✅ Succeeded
17 seconds
sbom-json-check

✅ Succeeded
19 seconds
sanity-optional-label-check

✅ Succeeded
20 seconds
sanity-label-check

✅ Succeeded
8 seconds
show-summary

Pipelines as Code CI/test-component-pac-wueg-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
6 seconds
init

✅ Succeeded
16 seconds
clone-repository

✅ Succeeded
29 seconds
build-container

✅ Succeeded
19 seconds
deprecated-base-image-check

✅ Succeeded
19 seconds
sanity-inspect-image

✅ Succeeded
11 seconds
clair-scan

✅ Succeeded
11 seconds
sbom-json-check

✅ Succeeded
37 seconds
clamav-scan

✅ Succeeded
8 seconds
sanity-label-check

✅ Succeeded
8 seconds
sanity-optional-label-check

✅ Succeeded
7 seconds
show-summary


text:
Appstudio update test-component-pac-ctcg

Pipelines as Code configuration proposal

Pipelines as Code CI/test-component-pac-ctcg-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
14 seconds
init

✅ Succeeded
16 seconds
clone-repository

✅ Succeeded
2 minutes
build-container

✅ Succeeded
42 seconds
deprecated-base-image-check

✅ Succeeded
42 seconds
inspect-image

✅ Succeeded
7 minutes
clair-scan

✅ Succeeded
2 minutes
clamav-scan

✅ Succeeded
1 minute
sbom-json-check

✅ Succeeded
20 seconds
optional-label-check

✅ Succeeded
19 seconds
label-check

✅ Succeeded
7 seconds
show-sbom

✅ Succeeded
9 seconds
show-summary

Pipelines as Code CI/test-component-pac-ctcg-on-pull-request has successfully validated your commit.

StatusDurationName

✅ Succeeded
13 seconds
init

✅ Succeeded
21 seconds
clone-repository

✅ Succeeded
36 seconds
build-container

✅ Succeeded
19 seconds
inspect-image

✅ Succeeded
19 seconds
deprecated-base-image-check

✅ Succeeded
1 minute
clamav-scan

✅ Succeeded
14 seconds
clair-scan

✅ Succeeded
16 seconds
sbom-json-check

✅ Succeeded
19 seconds
optional-label-check

✅ Succeeded
17 seconds
label-check

✅ Succeeded
14 seconds
show-sbom

✅ Succeeded
29 seconds
show-summary


text:
chore(deps): update registry.access.redhat.com/ubi8/openjdk-17-runtime docker tag to v1.20-3.1721231685

This PR contains the following updates:

Package
Type
Update
Change

registry.access.redhat.com/ubi8/openjdk-17-runtime
final
minor
1.13 -> 1.20-3.1721231685

Configuration
📅 Schedule: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).
🚦 Automerge: Disabled by config. Please merge this manually once you are satisfied.
♻ Rebasing: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.

[ ] If you want to rebase/retry this PR, check this box

This PR has been generated by Renovate Bot.

Pipelines as Code CI/konflux-demo-component-on-pull-request has successfully validated your commit.
Namespace: konflux-gfdt-tenant
PipelineRun: konflux-demo-component-on-pull-request-b457l

Task Statuses:

StatusDurationName

✅ Succeeded
7 seconds
init

✅ Succeeded
14 seconds
clone-repository

✅ Succeeded
1 minute
build-container

✅ Succeeded
1 minute
clamav-scan

✅ Succeeded
10 seconds
push-dockerfile

✅ Succeeded
39 seconds
clair-scan

✅ Succeeded
11 seconds
sbom-json-check

✅ Succeeded
10 seconds
deprecated-base-image-check

✅ Succeeded
5 seconds
apply-tags

✅ Succeeded
11 seconds
sast-snyk-check

✅ Succeeded
25 seconds
ecosystem-cert-preflight-checks

✅ Succeeded
6 seconds
show-sbom

✅ Succeeded
14 seconds
show-summary

Integration test for snapshot konflux-demo-app-msp4w and scenario my-integration-test-octg has passed

Pipelinerun: konflux-demo-app-msp4w-zcfwc

Task
Duration
Test Suite
Status
Details

task-success
5s

:heavy_check_mark: SUCCESS

task-success-2
5s

:heavy_check_mark: SUCCESS

task-skipped
6s

:white_check_mark: SKIPPED

repo: redhat-appstudio-qe/hacbs-test-project


text:
[DNM] Test PaC image update

Do not merge.
This is a test of the PaC image update to 0.17.4.

/test all

/hold

/test appstudio-e2e-tests

repo: redhat-appstudio/infra-deployments


text:
Add Global page charts

Add the two charts in the global page
Filter options will be add in diffrent PR
ChartJS: https://www.chartjs.org/docs/latest/

I will add tests for the query soon

I added the tests

repo: redhat-beyond/Xpense


text:
cert manager operator inconsistent namespace

The namespace field in base/kustomization.yaml should be updated to match the namespace.yaml resource (cert-manager-operator).
https://github.com/redhat-cop/gitops-catalog/blob/main/cert-manager-operator/operator/base/kustomization.yaml
I believe the startingCSV field in the base/subscription.yaml is superfluous and may be removed.

Thank you for pointing this out.

authors: adetalhouet dlbewley

repo: redhat-cop/gitops-catalog


text:
Update kube-rbac-proxy to v4.9

This is required for ARM, for which 4.9 is the initial version.

fixed by #26

authors: garethahealy yselkowitz


text:
chore: removed i18n from use commands

Closes #1032

The fact that the generated docs have not changed automatically verifies this one :)

authors: alexal craicoverflow

repo: redhat-developer/app-services-cli


text:
Context Feature Branch

Feature branch. Do not merge

@bhardesty flagging this PR for review. We can create jira/discuss that on backlog refinement

rhoas context use-service-registry   --name is missing suggestions that we have for kafka.

@bhardesty flagging this PR for review. We can create jira/discuss that on backlog refinement

Ack. I was planning to review/edit the command help. Should I do that in a separate PR once this is merged?

Any way that would work for you best. I think we will keep this as feature branch for some time (at least 1-2 weeks) to make sure that we do not have any issues

rhoas context use-service-registry --name is missing suggestions that we have for kafka.

Suggestions/completions are missing in original service-registry use command. WE should wait till api allows filtering registries.

WE should wait till api allows filtering registries.

We can also show all registries no matter what user typed

authors: bhardesty rkpattnaik780 wtrocki


text:
fix(orchestrator): pass businessKey for assessments

backport of https://github.com/janus-idp/backstage-plugins/pull/2558

@mareklibra
to fix prettier run:
cd workspaces/orchestrator
yarn prettier --write .
then commit the changed files

The prettier is passing when executed in the plugin's folder but not on the workspace level.
We have discrepancy in the dependencies for prettier.
I will try to fix that in a separate PR.

meanwhile can you fix it so we can get this merged?
I need to backport https://github.com/janus-idp/backstage-plugins/pull/2573 and it would be difficult without this PR merged

authors: batzionb mareklibra

repo: redhat-developer/rhdh-plugins


text:
Add material/patternfly ui for function workflow

create a custom webview to guide function workflow. Use patternfly for the forms and components creation.
Patternfly: https://www.patternfly.org/v4/components/form/

vscode-wizard is the recommended way to built forms in VSCode. It has been used by several of our plugins (Kafka, RSP, Tekton). It should be the way to go unless there are good reasons to deviate

authors: jeffmaury mohitsuman

repo: redhat-developer/vscode-knative


text:
separate tasks to power off nodes with ipmi and redfish

Fixes #155

check dallas ocp-4.14-vanilla

/lgtm

check dallas ocp-4.14-vanilla

Looks like libvirt job got stuck before reaching the task under test, can we repeat that test to confirm it's working?

Hi Ramon, good observation, I checked the libvirt job and actually what it got stuck was the output in the DCI console, but if you look at the ansible.log file all it's in there[1]. So I think it's more like a DCI issue, because it's not showing anything after the task Check if override path is defined for customize_filesystem and that's like 20 tasks before the task that powers off the nodes. But again the full output it's on the ansible.log
[1]
2024-02-01 15:26:11,664 p=1161722 u=dci-openshift-agent n=ansible | TASK [redhatci.ocp.installer : Power off hosts via IPMI] ***********************
2024-02-01 15:26:12,216 p=1161722 u=dci-openshift-agent n=ansible | ok: [provisionhost.server09.partnerci.bos2.lab] => (item=dciokd-master-0) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": "dciokd-master-0", "powerstate": "off", "warnings": ["The value False (type bool) in a string field was converted to 'False' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change."]}
2024-02-01 15:26:12,591 p=1161722 u=dci-openshift-agent n=ansible | ok: [provisionhost.server09.partnerci.bos2.lab] => (item=dciokd-master-1) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": "dciokd-master-1", "powerstate": "off", "warnings": ["The value False (type bool) in a string field was converted to 'False' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change."]}
2024-02-01 15:26:13,052 p=1161722 u=dci-openshift-agent n=ansible | ok: [provisionhost.server09.partnerci.bos2.lab] => (item=dciokd-master-2) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": "dciokd-master-2", "powerstate": "off", "warnings": ["The value False (type bool) in a string field was converted to 'False' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change."]}
2024-02-01 15:26:13,446 p=1161722 u=dci-openshift-agent n=ansible | ok: [provisionhost.server09.partnerci.bos2.lab] => (item=dciokd-worker-0) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": "dciokd-worker-0", "powerstate": "off", "warnings": ["The value False (type bool) in a string field was converted to 'False' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change."]}
2024-02-01 15:26:13,829 p=1161722 u=dci-openshift-agent n=ansible | ok: [provisionhost.server09.partnerci.bos2.lab] => (item=dciokd-worker-1) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": "dciokd-worker-1", "powerstate": "off", "warnings": ["The value False (type bool) in a string field was converted to 'False' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change."]}
2024-02-01 15:26:14,210 p=1161722 u=dci-openshift-agent n=ansible | ok: [provisionhost.server09.partnerci.bos2.lab] => (item=dciokd-worker-2) => {"ansible_loop_var": "item", "attempts": 1, "changed": false, "item": "dciokd-worker-2", "powerstate": "off", "warnings": ["The value False (type bool) in a string field was converted to 'False' (type string). If this does not look like what you expect, quote the entire value to ensure it does not change."]}
2024-02-01 15:26:14,210 p=1161722 u=dci-openshift-agent n=ansible

authors: derekhiggins manurodriguez