text:
Ghost types

Not sure if here is the right place, but there are some ideas for filling const generic params with runtime values in ZSTs, that enables dependent types for "proof as program" style things.
Ghost types
A type is ghost if it has no runtime data and effect. That is, a ZST annotated with a ghost keyword:
ghost struct Le<const A: usize, const B: usize>(());
// we control instantiating of this type in a way that it has instance iff A <= B

Why someone would annotate type with ghost? Because it will enable for its const generic parameters to depend on runtime values, so this will be accepted:
let x = 2;
let prf: Le<x, 5> = Le::<x, 5>(());

The x in Le<x, 5> will be considered as a const variable, and only unifies with itself. only immutable variables and function parameters are allowed (that types doesn't contain mutable reference or interior mutablity) to be in const generics of ghost types.
This allows to write function with dependent signature:
fn do_smth(x: usize, y: usize, x_le_y: Le<x, y>) {
  // we know x is lesser than y, so we can for example index an array without bound checking.
}

Which is the whole propose of ghost types. Benefits of powerful type system is clear, it can enable more correct and specified code, less unsafe code, less runtime checks, ... This kind of thing is actually the tool for formal verification in languages like Coq. I don't explain it more as you are probably already aware of.
Type checking
How we type check such function calls? It will reuse const generic infrastructures. For example that function will become roughly equivalent to this for type checking propose:
fn do_smth<const x: usize, const y: usize>(x_le_y: Le<x, y>);

And in call site, we consider each immutable binding as a const variable and each parameter as an abstract const expression, so in:
do_smth(f(a), g(2), prf);

f and g should be const fn
if we assume g(2) is 7, type of prf should unify with Le<f(a), 7>.

so this is actually different with:
let b = f(a);
do_smth(b, g(2), prf);

as here f can be anything and type of prf is Le<b, 7> in this case.
Code generation
After type checking, we replace all ghost types with (), so that function will become:
fn do_smth(x: usize, y: usize, x_le_y: ());

which is normal function today.
I have some extenstions like ghost fn in mind, but I think this is enough for starting discussion.

How we type check such function calls? It will reuse const generic infrastructures. For example that function will become roughly equivalent to this for type checking propose:
fn do_smth<const x: usize, const y: usize>(x_le_y: Le<x, y>);

and

After type checking, we replace all ghost types with (), so that function will become:
fn do_smth(x: usize, y: usize, x_le_y: ());

seems to disagree with each other.

More generally, while I would love a language like Rust with dependent types, I do not believe that we should add them to Rust itself.
There are three reasons for this:

adding it to Rust without any breaking changes will leave us with large parts of std/the ecosystem at large not supporting it, which makes using it a lot more cumbersome than needed
dependent types really do not mesh well with our current query system and would need fairly large changes to how typeck and const evaluation works. Even feature(generic_const_exprs) feels like its pretty much at the edge of what we can support.
const fns are not guaranteed to evaluate successfully, so you can't use fn(MyStatement) -> ! as a proof of ¬MyStatement and I don't believe that adding something like ghost fn or pure fn for this is worth it. To my knowledge this is an important part of any useful proof assistant.

Other members of the language or compiler team might be a bit more open to something like your proposal, but even then, I do not assume this to be something we're going to tackle any time soon.
An imo far more promising approach would be unique lifetimes outside of closures. While using them will be more complex than actual dependent types, these lifetimes can already be used to assert things about the runtime behavior of your programs, e.g. https://twitter.com/lcnr7/status/1390653250383355904

seems to disagree with each other.

Can you explain more? That just means we type check ghost types as const generics, but remove them before monomorphisation and code generation.

adding it to Rust without any breaking changes will leave us with large parts of std/the ecosystem at large not supporting it, which makes using it a lot more cumbersome than needed

There are already three style of api: panicking, fail with Option/Result, and unsafe. A forth style that takes proofs as input can be added?

dependent types really do not mesh well with our current query system and would need fairly large changes to how typeck and const evaluation works. Even feature(generic_const_exprs) feels like its pretty much at the edge of what we can support.

Dependent types (types that depends on value) are already here, but they are limited to compile-time constant, and it isn't because typeck (Correct me if I'm wrong), it is because compiler can't generate code for [i32; runtime_value] and need to monomorphize types for code generation. If we limit usage of types that depend on runtime values to ZSTs, we can remove them in code generation. Ghost type is all about it. It shouldn't make difference in type checking,

const fns are not guaranteed to evaluate successfully, so you can't use fn(MyStatement) -> ! as a proof of ¬MyStatement and I don't believe that adding something like ghost fn or pure fn for this is worth it. To my knowledge this is an important part of any useful proof assistant.

fn(MyStatement) -> ! guarantees that ¬MyStatement is partially correct, i.e. programs that use it to proof other properties will work as intended or won't terminate (successfully). And it is not special to fn(MyStatement) -> !. Existence of Le<2, 5> only proofs that Le<2, 5> is partially correct, as its proof can come from let x: Le<2, 5> = todo!(). So it can't be used for proofing mathematics statements, but it is fine for rust programs. An annotation for total functions is useful (even for non-const functions and regardless of ghost types) but it is another topic.

Can you explain more?

if we typeck these as const parameters, they can never be used with runtime values, so there isn't really a need to codegen them at all, they could just be const fn's or even associated consts of a generic type.

There are already three style of api: panicking, fail with Option/Result, and unsafe. A forth style that takes proofs as input can be added?

we not just want things to take proofs as inputs, we also want to add proofs of postconditions as outputs, don't we? These three kinds of api only rarely exist for the same functionality, so I believe you underestimate the necessary amount of duplication. Esp. because the "proof" versions are pretty much a strict improvement over non proof methods, so having both seems suboptimal.

Dependent types (types that depends on value) are already here, but they are limited to compile-time constant, and it isn't because typeck (Correct me if I'm wrong)

To me, the core part of dependent typing is that values from a context can flow into types in that same context. Rusts implementation of const generics seperate computations used in types from the places where these types are used which makes it a lot easier to querify. Right now we can separately compute any const arguments during the typeck where these constants are used. Once these values are used in types which exist in the same context as they do themselves, the separation between const eval and typeck breaks down. We then have to partially evaluate a function while typechecking it which is far from trivial.
Finally, these statements seem contradictory

fn(MyStatement) -> ! guarantees that ¬MyStatement is partially correct, i.e. programs that use it to proof other properties will work as intended or won't terminate (successfully).

After type checking, we replace all ghost types with ()

if we replace all ghost types with () after typeck we have to catch all "false proofs" during compile time, for a function like
fn proof_of_not_P(x: &str) -> fn(P<x>) -> ! this seems pretty much impossible to me.
This is at the edge of my knowledge, but more generally: for ghost types to be erasable before codegen you need proof irrelevance, i.e. the values of your proof objects must not flow into non proof objects. I am not sure how one would add proof irrelevance to rust.

I believe this is related to The pi type trilogy, especially Fully dependent pi types..

Esp. because the "proof" versions are pretty much a strict improvement over non proof methods, so having both seems suboptimal.

They are more cumbersome to use so it isn't improvement in this sense, and because of this most people won't use the proofed versions. So existence of normal version isn't bad imo. Unsafe codes, crictical code and library implementations can use it. Normal versions can be a wrapper over proofed versions.
That said, I understand your concern. Specially, good names are reserved for non proofed version, and this can make using proofed versions even more cumbersome. But the ability of express proofs is still very useful imo, even if it won't be used by default.

Finally, these statements seem contradictory

Function pointers are not zst, so they can not be (part of) ghost types. Whole propose of declaring a type as ghost is to using runtime values for its const generic parameters, not marking a type that will be used as proof. Another type that isn't ghost but can be used as proof is this:
pub enum LogicalOr<A, B> {
    Left(A),
    Right(B),
}

If A and B are ghosts, this still has one bit of data and can affect runtime behavior. We don't remove it (nor function pointers) from runtime. If someone wants to use fn (T) -> ! to proof something else, they need to actually call it, and if it is wrong, we may reach that call and panic (or loop), but no bad thing will happen. For example:
fn example(a: usize, b: usize, not_a_le_b: fn(Le<a, b>) -> !) -> Le<b, a> {
    match decide_le(a, b) {
        Left(a_le_b) => not_a_le_b(a_le_b),
        Right(b_le_a) => b_le_a,
    }
}

If proof of not_a_le_b is cheat, and b_le_a is wrong, we will see panic (or non termination).

This is at the edge of my knowledge, but more generally: for ghost types to be erasable before codegen you need proof irrelevance.

Ghost types are zst, and each two instance of a zst type are equal. For other proofs, there is no proof irrelevance, and we don't erase them.

if we typeck these as const parameters, they can never be used with runtime values, so there isn't really a need to codegen them at all, they could just be const fn's or even associated consts of a generic type.

We consider each function parameter and let binding a const variable, build a new function for each function, typeck these functions, then remove these functions and generic parameters of ghost types, and continue as normal. This type checking is only a check, like borrow checking, and won't affect anything else.
For example:
fn f1(x: usize) -> P<x> {
    todo!();
}

fn f2(a: usize, b: P<a>) -> bool {
    let c = f1(a);
    c == b
}

fn f3() {
    let x = 2;
    let px = f1(x);
    let b = f2(x, px);
}

At first we type check this code:
fn f1<const PARAM_x: usize>(x: usize) -> P<PARAM_x> {
    todo!();
}

fn f2<const PARAM_a: usize>(a: usize, b: P<PARAM_a>) -> bool {
    let c = f1::<PARAM_a>(a);
    c == b
}

fn f3<const LOCAL_x: usize>() {
    let x = 2;
    let px = f1::<LOCAL_x>(x);
    let b = f2::<LOCAL_x>(x, px);
}

And if it succeed, we completely ignore them and go with this:
fn f1(x: usize) -> P {
    todo!();
}

fn f2(a: usize, b: P) -> bool {
    let c = f1(a);
    c == b
}

fn f3() {
    let x = 2;
    let px = f1(x);
    let b = f2(x, px);
}

We then have to partially evaluate a function while typechecking it which is far from trivial.

Ghost types don't change how we unify const generics, so if 1+N won't be unified with N+1, 1+runtime_x won't be unified with runtime_x+1 as well. This harms usability of proofs, but can be worked around with axiomatization.

Looked more into dependent types and took some time to think more about the current status of const generics. I disagree with some of my previous statements here. That is less than ideal, considering that I am the lead of the const generics effort. I wasn't as thoughtful of my authority as I should have, sorry for that.
One of the more interesting things I've looked it is an accepted proposal for adding dependent types to Haskell.
While I still believe that this issue is something we don't have the capacity for right now and I don't intend to spend a lot of time on this subject at least until a basic form of #![feature(generic_const_exprs)] is close to stable, I am going to quickly summary my current position here. I am not yet an expert on dependent types, so that opinion may again change in the future.

adding the abilities for locals and function parameters to be influence types is something we may add in the future. Given infinite resources we should add it.

I don't want to work towards it or flesh out the design for that right now, as there are a lot of - imo more important - parts of const generics we have to finish first.
I intend to be careful when stabilizing features so that they do not prevent us from adding this in the future.

Adding ghost types does not seem like the right approach to me. The actual things getting erased should be generic parameters, not types. Guaranteeing that a generic parameter does not impact the runtime behavior is fairly straightforward and something I am interested in. This is also related to polymorphization, where this erasure happens implicitly.

This is again something I do not have the capacity for rn.

I intend to open a new design doc for "allow locals and function parameters to be used in types" which should contain a summary of this issue in the near future. At that point I will close this issue so that only design docs are open in this repository. I prefer zulip for active discussions.

I also lost my interest on this, particularly because it would not work for proofing properties of the mutable, imperative code, which is idiomatic and performant rust. That said, it would be still helpful in encoding properties at api boundary in the type system.

The actual things getting erased should be generic parameters, not types.

We can simulate erasing generic parameters by wrapping them in some ghost type, so there is some equivalency, I think annotating types is better than annotating generic parameters if it needs some explicit annotation, but both would work anyway.

I don't intend to spend a lot of time on this subject at least until a basic form of #![feature(generic_const_exprs)] is close to stable

And probably dependent const generics, i.e. X<const A: usize, const B: Foo<A>> needs to be fully implemented/near stabilization, which needs ADT in const generics as well (I don't know their feature flags). So yeah, it is not actionable, and summarizing into a design doc and closing this is a good outcome for now.

authors: HKalbasi lcnr meowjesty

repo: rust-lang/project-const-generics


text:
[WIP] Add random testing of crates.io regex

This PR implements the first half of #468.
I've marked it as a WIP until I get to the bottom of the \b find_iter bug.
The tests this PR adds take a long time to run, so I disabled them by default, but added logic to ci/script.sh to run them during a travis.ci cron job. I think someone with access to the travis console will need to enable that (so probably @BurntSushi or @alexcrichton). I'm also totally happy to rip the ci changes out or just leave the tests always enabled.
Commit Msg:
This patch adds some infastructure to scrape crates.io
for regex, then run each of the regex found in this way though
a random testing gauntlet to make sure that all the different
backends behave in the same way. These random tests are
expensive, so we add a feature gate around them so that
normal CI does not get bogged down. If the CI tests are being
run in a cron job, we trigger the random testing.
To make this better we should add something which can generate
a matching string from a regex. As is this will just focus on
the negative case.

Thanks for the review!
I tried to address the comments in the patch I just pushed.
This PR is still blocked on the word boundary bug, I'll try to get to that soon.
WRT time it takes about 35 seconds in release mode and 9ish minutes in debug mode. Wow that makes a difference! I think I should probably just ditch the ci shenannigans and enable release mode for the random testing every time.

I think I should probably just ditch the ci shenannigans and enable release mode for the random testing every time.

Sounds good!

This PR is still blocked on the word boundary bug, I'll try to get to that soon.

If this turns out to be hard (and it may very well be), would it be possible to comment out those tests for now in the interest of landing the PR? I think this is overall great work, and I don't want to put you in a position where you need to fix existing bugs before landing it.

If this turns out to be hard (and it may very well be), would it be possible to comment out those tests for now in the interest of landing the PR? I think this is overall great work, and I don't want to put you in a position where you need to fix existing bugs before landing it.

Definitly! The last bug turned out the be pretty shallow when I actually looked at it, so I'm hoping this will be the same. If it leads to lazy DFA arcana I won't hesitate to bail out.
Do you happen to know if there is an easy way to enable release mode for just one test binary? If there isn't I can probably break out the build.rs hackery, but that seems gross to me.

Not to my knowledge unfortunately. build.rs hackery sounds like the best bet.

I had trouble with the word boundary bug, so I commented it out. I've removed the WIP status.
I also had trouble figuring out a build.rs hack because it looks like you are not allowed to specify arbitrary rustc flags. Instead I went back to the environment variable and CI script approach. The difference is that I no longer special case cron jobs. The random tests are run in release mode on every CI run.

I rebased this patch on top of master. Github is telling me there are changes requested, but I think the patches those comments were attached to got lost to rebasing. I do remember taking steps to address all of them though. If there is anything else I can do, please let me know!

On my machine it takes 3:36 of wallclock time (though 17:30 of CPU time). Milage may vary on the different CI servers, but I think that the startup time of AppVeyor will probably still dominate.

@ethanpailes Hmm. I'm not getting the answer I'm looking for, so I must be asking the wrong question.
Forget about CI for a moment. Adding a couple minutes to an already 10 minute build is probably something I can live with, and we don't need to make it conditional.
What I care about a lot more is the cycle time during development. There are three different things to test here:

Compile and run all tests.
Compile all tests, run everything except for the crates.io tests.
Do not compile the crates.io tests at all, but compile and run everything else.

This is the kind of thing I do a lot when working on the regex crate, so it's important to keep this cycle time down. I don't know which option to pick though because I don't have the data.
When I have time, I can pull down this PR and figure out the data points if this is still unclear.

That makes sense. I'll try to provides some times that outline of various OODA loops with notes on how I personally use each that may or may not be useful.

Action
Time
Notes

cargo clean && cargo test
1:49
This one probably does not matter that much, but it is still worth knowing. cargo test alone will not actually run the crates.io tests because the RUST_REGEX_RANDOM_TEST environment variable is not set.

After a clean build insert println!("a change") into src/exec.rs:ExecBuilder::new and cargo test
1:14
I run this sort of thing when I'm finalizing a patch. Again, will not run the crates.io tests.

After a clean build insert println!("a change") into src/exec.rs:ExecBuilder::new and cargo test --test default
0:15
This sort of thing is my main driver and probably should not have changed.

After a clean build remove println!("a change") from src/exec.rs:ExecBuilder::new and export RUST_REGEX_RANDOM_TEST=1 && time cargo test --release --test crates-regex
0:48
This will run the crates.io tests, and is faster than compiling in debug mode. Does not run any other tests.

./ci/script.sh
3:36
This one compiles and runs all tests, including the crates.io tests. I usually do this when finalizing a patch. The best way to compile and run all commands is with two commands (one for the crates.io tests in release mode, and one for the rest in debug mode).

I don't have a time for (2) from your list, because I don't know of a negative test filter you can pass to cargo test, so the only way I can think to do it is to enumerate all the other tests, which seems annoying.

I don't have a time for (2) from your list, because I don't know of a negative test filter you can pass to cargo test, so the only way I can think to do it is to enumerate all the other tests, which seems annoying.

I think it's actually the second entry in your table, no? That is, it compiles all tests, but runs everything except for the crates.io tests.
I think the thing you don't have is how long cargo test takes without needing to compile the crates.io tests.
With that said, it's a good point about cargo test --test default. That does tend to be the main thing that I run, but I do just run cargo test a bit as well.

I think it's actually the second entry in your table, no?

You're right. Derp.

I do just run cargo test a bit as well

1:15 is something I'd personally be comfortable with for an outer feedback cycle (I might have been warped by 15 minute C++ outer feedback cycles), but it is definitly starting to become annoying. I think ideally we would configure cargo test with no extra args not to compile the crates.io tests. I ran out the steam on that last time, but I can take another crack at it if you want.

It's fine. Let's just merge this, and if I become annoyed, I'll fix it. :-)

Thanks again!

Thanks!

source: gharchive/pull-request

authors: BurntSushi ethanpailes

repo: rust-lang/regex

license_type: permissive

license_source: bigquery


text:
Special lifetime for downcasting ownership to reference.

Suppose I have a trait:
pub trait Summarize {
    type Summary;
    fn summarize(self) -> Self::Summary;
}

Now, for my type, summary is non-mutative, but does reference the original. So I have a struct:
struct MySummary<'a> {
    obj: &'a MyType,
    /* and other things */
}

and the helper summary function I implemented uses a reference:
impl MyType {
    fn do_summarize<'a>(&'a self) -> MySummary<'a> { /* do it */ }
}

Now I implement the Summarize trait for references:
impl<'a> Summarize for &'a MyType {
    type Summary = MySummary<'a>;
    fn summarize(self) -> MySummary<'a> { self.do_summarize() };
}

But I also want to implement it for owned copies. Currently, I can only reimplement everything with a struct that owns the object, a method that returns such a struct, and then implementation of the trait. What I would like to be able to do is:
impl Summarize for MyType {
    type Summary = MySummary<'owned>;
    fn summarize(self) -> MySummary<'owned> { self.do_summarize() };
}

with the special lifetime 'owned assuming the lifetime of the ownership, which is also the lifetime of the reference implicitly created by that last self.do_summarize() call. And similarly for downcasting from ownership to a &mut.

The memory representation of &'a MyType is a pointer, which is probably different from an owned, inline MyType. And a lifetime parameter cannot change the memory representation of your struct.
I think you’re looking for std::borrow::Cow, an enum that contains either a borrowed (with a &_ reference) thing or an owned thing. Replace &'a MyType in the struct MySummary<'a> definition with Cow<'a, MyType>.
In the second impl, use the MySummary<'static> and create an owned cow: Cow::Owned(something). The 'static lifetime is special and represents the entire duration of the program’s execution. Any reference inside MySummary<'static> must be valid that long. But since you’re not using a Cow::Borrowed variant, there isn’t any such reference.

Thanks! This does indeed solve the code duplication, so it might render this feature useless. I still think using a Cow might be an overkill (with some overhead), so I'll try to elaborate:
When do_summarize is called on a borrowed reference, the named lifetime 'a in its definition gets set to the lifetime of the said reference, and the same happens downstream for the struct MySummary<'a>. This means that when I have an obj: &'a MyType, I know that the return type of obj.do_summarize() will be MySummary<'a>.
On the other hand, when do_summarize is called on an owned object, a reference is still generated, since the signature of the function demands it. However, the lifetime of this implicit borrow is something I cannot name outside of the function. Therefore, if I have an obj: MyType, I cannot write out the return type of obj.do_summarize(), though it is still MySummary<'something>, holding a reference.
I will try to think of another example where this is more hindering.

As stated, there is a conflict in your problem statement :  Your fn summarize(self) -> Self::Summary necessarily destroys self, which works fine if Self is a reference.  If Self is MyType however then you cannot return a MySummary that references destroyed data.  You could include an Cow<'a, MyType> in MySummary but that sounds needlessly complex.
I suspect you want a trait more like
pub trait Summarize {
    type Summary;
    fn summarize(&self) -> Self::Summary;
}

impl Summarize for MyType where {
    type Summary = MySummary;
    fn summarize<'a>(&'a self) -> MySummary<'a> { .. }
}

At least the [https://doc.rust-lang.org/nomicon/lifetime-elision.html](lifetime elision) rules say the fn becomes the desired fn summarize<'a>(&'a self) -> MySummary<'a> { .. }.  And maybe this says that type Summary = MySummary works too.

I checked, and it doesn't work, because that lifetime parameter is really required there in type Summary = MySummary.
The thing is, that my original fn summarize(self) -> Self::Summary doesn't destroy the data, it just moves the ownership. And I would like, for the sake of avoiding code duplication, to be able to "downgrade" my ownership into just a ref / mut ref. And then, the data still exists there until it runs out of scope, like with regular ownership, but it is only accessible via that ref.

You'll notice this errors with x "does not live long enough" even though I explicitly require X: 'a
fn f<'a,X>(x: X) -> &'a X where X: 'a {
    &x
}

so the same applies to self in your fn summarize(self) -> Self::Summary.  I'd expect LLVM to reuse the space self occupied when summarize returns.
I have used types with lifetime parameters used as associated types once before.  I cannot find my code example right now, but I think exploring the Cow<'a, B> type's bound B: 'a helped me do it.  You might explore some variations on :
impl<'a> Summarize for MyType where MyType: 'a {
    type Summary = MySummary<'a>;
    fn summarize(&'a self) -> MySummary<'a> { .. }
}

I believe this is going to be addressed by Generic Associated Types in https://github.com/rust-lang/rust/issues/44265.
trait Summarize {
    type Summary<'a>;
    fn summarize(&'a self) -> Self::Summary<'a>;
}

struct MyType;

struct MySummary<'a> {
    obj: &'a MyType,
}

impl Summarize for MyType {
    type Summary<'a> = MySummary<'a>;
    fn summarize(&'a self) -> MySummary<'a> {
        MySummary { obj: self }
    }
}

For now the closest stable workaround would be:
trait Summarize<'a> {
    type Summary;
    fn summarize(&'a self) -> Self::Summary;
}

struct MyType;

struct MySummary<'a> {
    obj: &'a MyType,
}

impl<'a> Summarize<'a> for MyType {
    type Summary = MySummary<'a>;
    fn summarize(&'a self) -> MySummary<'a> {
        MySummary { obj: self }
    }
}

or:
trait Summarize {
    type Summary;
    fn summarize(self) -> Self::Summary;
}

struct MyType;

struct MySummary<'a> {
    obj: &'a MyType,
}

impl<'a> Summarize for &'a MyType {
    type Summary = MySummary<'a>;
    fn summarize(self) -> MySummary<'a> {
        MySummary { obj: self }
    }
}

Closing per notes in https://github.com/rust-lang/rfcs/issues/1802#issuecomment-344468904.

source: gharchive/issue

authors: Centril SimonSapin amosonn burdges dtolnay

repo: rust-lang/rfcs

license_source: github-api


text:
Make mem::size_of_val and mem::align_of_val take raw pointers instead of reference

This would allow you to use null pointers to get the size of a value. As far as I know, these methods do not actually dereference  the &T that is passed in. Here is a gist and a  playground link that shows that they work just fine with null pointers, as long as you cast them to references so that it will type check.
I can't think of any reason that the pointer being passed in would actually need to be dereferenced. My understanding is that the pointer itself, along with its type, are all that are needed to give the size and alignment information.
Changing the signature to the following should be possible without breaking any backward compatibility, because &T is implicitly coerced to *const T.
fn size_of_val<T>(*const T) -> usize;
fn align_of_val<T>(*const T) -> usize;

Somewhat-relevant RFC: https://github.com/rust-lang/rfcs/pull/1524 (Custom DSTs)
cc @nikomatsakis @ubsan

You are saving vtable and reconstructing your reference with it, but it's the vtable that actually contains size information, so your example doesn't show anything. It works because you explicitly preserve the part that holds the size. Does null raw pointer have a vtable, or a slice length? I strongly doubt that.

@le-jzr Hmm... I didn't think about just calling ptr::null<Foo>() directly. You're right, there's no way the vtable would point to anything relevant.
Still, even if it's not a good idea for those functions to take *const T, it would be nice if there was a guarantee in the documentation that the &T doesn't actually have to point to anything valid, so that you could get the size and alignment info using just the vtable/slice length/other metadata. Unless there actually is some reason to dereference it, which I am not aware of.

But it needs to point to something valid, that's the whole point. You are querying the size of that "something valid". If you don't want to query a valid object, you can use mem::size_of::<T>().

You can't use size_of for ?Sized types - that's the whole point of size_of_val.

How is that supposed to work for array (slices)? It looks like in that case you are required yo use a fat pointer, otherwise it is impossible to know the size.

What does it mean to get size_of_val of unsized type without having a valid dereferencable pointer? That's meaningless concept, you either have a pointer you can dereference to a valid value, or you don't have any value at all and it's nonsense to query size of something that doesn't exist.

In std::ptr, pub const fn null<T>() -> *const T { 0 as *const T } does not have a ?Sized bound. It cannot be used to create a null raw slice or null raw trait object.

We haven't defined whether raw unsized pointers need to have valid metadata, I don't think.
@SimonSapin You can go further than that and just try 0 as *const Trait outside of a generic method - it's not allowed at all, although it's not clear what the intention behind that was.

If you want to allow a null pointer, just use mem::size_of, since you can only construct a null pointer for Sized values. mem::size_of_val for raw pointers is not a completely pointless usecase, but it would be for when you have a pointer that you don't know for sure points to valid memory rather than for null pointers.

I've been thinking about this again recently. As some of you have pointed out, ptr::null and null_mut have a Sized bound, so it is impossible to produce a pointer to a dynamically sized type   (DST) that has invalid metadata (meaning length for slices, or vtable for trait objects) without unsafe code.
In fact, the only way to get a null pointer to a DST is to cast from a sized type, e.g. ptr::null() as *const SomeTrait. @ranma42 and @le-jzr does that answer your questions/concerns?
One concern that might come up, is the possibility that raw pointers to DSTs might one day possibly have a thin representation, where the pointer would have to be dereferenced to get at size/alignment info in the metadata. If thin pointers are implemented, though, I'm pretty sure they would have to be (or at least, really should be) a separate type from *const and *mut T. I've seen repr(thin) being floated around, but there's no reason that a type should have to choose between "fat" or "thin" when it can have both.

@mikeyhew currently the as cast you are suggesting is disallowed:
pub fn example() -> * const [i32] {
    let x: * const u8 = std::ptr::null();
    x as * const [i32]
}

fails with error[E0607]: cannot cast thin pointer *const u8 to fat pointer *const [i32].
Regarding my concerns, it is still not clear to me how to provide the number of elements of the slice when performing the cast (unless you manually and unsafely construct the pointer/length pair and transmute it to a slice).

@ranma42 you need to cast from a type that implements Unsize<[i32]>, such as a fixed size array:
let slice = ptr::null::<[i32; 3]>() as *const [i32];

The way it works is, the compiler provides the implementation  of Unsize<[i32]> for [i32; 3], and uses that implementation when doing the coercion from *const [i32; 3] to *const [i32].

authors: SimonSapin Vurich eddyb le-jzr mikeyhew ranma42


text:
Using invalid attributes should be an error

I was very surprised when I learned that not only is using an invalid attribute not an error, but it is also only a lint, meaning that it can be disabled. That means that if a new attribute were added post-1.0, it would potentially break existing code. Is there any reason that this is not already an error with no option to disable it?

Triage: fixed (now hidden behind a custom_attribute feature).
/cc @alexcrichton

Yup!

authors: P1start Stebalien steveklabnik


text:
Add read_into_buf and get_buf to BufRead

Add two methods to BufRead, read_into_buf and get_buf. These would allow filling up more of the buffer, and inspecting the buffered data without causing an implicit read.
Rendered
cc @aturon @alexcrichton @sfackler

I always assumed fill_buf to act as your read_into_buf + get_buf together. I guess I was lucky that this didn't bite me yet.

An implementation is currently used in hyper. It was unfortunate needing to copy the BufReader from std, since otherwise the buf field is private and these methods couldn't be implemented as a BufExt trait or whichever.

This is a particularly useful pattern for high performance parsing. For example, one of the tricks rapidjson does to speed up parsing is to use SIMD instructions to parse N whitespace characters at a time. We could use this trick if we could get access to the underlying BufRead buffer (and of course my long-desired IntoBufferedRead trait :smile:).

@erickt that's pretty much my exact usecase (except parsing HTTP instead of JSON).

I think this RFC as written proposes a backwards incompatible change, yes? Existing implementations of BufRead would break with the addition of non-default methods.
Is there a way to salvage the RFC? (I don't see any obvious way to provide default impls of these methods.)

I don't believe a default could be provided for these methods. To salvage this, it would need... to be a new trait? trait BufReadExt: BufRead {}?

This RFC is now entering its week-long final comment period.

I agree with @seanmonstar's analysis that this can no longer be implemented (due to stabilization), and it doesn't seem worth it to add an extra trait at this time.

Something I thought of also, would be to provide default implementations with unimplemented!(). If there was a way to add a warning for implementors that they should override the default, then it will prevent breakage while poking people to implement it.
This solution would allow adding methods to any trait.

I've wanted this exact functionality myself, so :+1: to the idea. It'll be nice if we can find a way to add this functionality in a backward-compatible way, but if not, I think this should be a strong contender for adding in 2.0.

The consensus of the libs team was to close this RFC at this time. We think that extension traits should bake outside the standard library first, and otherwise it looks like this RFC may want to start anew with a fresh look at backwards-compatibility. Thanks regardless for the RFC @seanmonstar!

authors: BurntSushi alexcrichton erickt oli-obk rkjnsn seanmonstar


text:
adding config for cargo-bitrig-64

I think this correctly adds the configuration for cargo-bitrig-64.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @brson (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. The way Github handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see CONTRIBUTING.md for more information.

Thanks! I'll deploy this tomorrow.

Hrm...I looked more closely and it looks like we only gate on builders with names that start with "auto": https://github.com/rust-lang/rust-buildbot/blob/master/master/master.cfg#L1824
The BSKiller class takes a lambda filter that only considers builds with names that start with "auto" and are not in the nogate_builders list.
So my change to the nogate_builders list doesn't do anything.

I just removed the change to nogate_builders since it doesn't do what I think it does.

@brson status?

I'm going to deploy this with some other changes tonight.

Deployed.

authors: brson dhuseby rust-highfive

repo: rust-lang/rust-buildbot


text:
Transcribe the Compiler Lecture oli-obk on miri and constant evaluation

Video: https://www.youtube.com/watch?v=5Pm2C1YXrvM

@rustbot claim

repo: rust-lang/rustc-guide


text:
Fix is_subpath

Closes #3627.

thanks!

Thank you! And glad I was able to be of some assistance 👍
I was also thinking it would be good to augment the existing tests to catch this "no system/idempotence tests ran" scenario to prevent a future recurrence. The existing test cases include a check to ensure there were 0 failed tests, so perhaps something as simple as adding another check that ensures some minimum number of tests ran/passed could address that.
Do folks think that would be valuable/helpful? If so I'd be happy to submit a PR

@calebcartwright that'd be nice to have, to catch such things in the future!

authors: calebcartwright scampi topecongiro

repo: rust-lang/rustfmt


text:
Return Result from Shape methods

The Shape modifier methods now return Result<Shape, ExeedsMaxWidthError>
There are _opt variants of the methods for when an Option is still needed

I believe this will remain useful in the long term, since sometimes we want to fallback and never error out

There is a minor change in behavior for when multiple methods are chained together - the width used in the error will be the last successfully modified width. I think this is okay?

Oops I accidentally opened this and it's not quite ready. But now you can see my plan and comment if you like!

IMO, using the last successfully modified width is fine. Actually, that might be the more accurate width in some cases.

IMO, using the last successfully modified width is fine. Actually, that might be the more accurate width in some cases.

by the way @ding-young thank you for all your excellent work this year! I know you worked closely with @ytmimi and we didn't get a chance to interact much, but i really appreciate all the work you did and I'm glad that you have expressed an interest in remaining engaged with it

There is a minor change in behavior for when multiple methods are chained together - the width used in the error will be the last successfully modified width. I think this is okay?

@camsteffen - re: this item, i've not looked at the diff yet but if there's a change to the formatting output (even if it's an improvement) then we'll need to edition gate it behind style edition 2027
e.g.
https://github.com/rust-lang/rustfmt/blob/777e25a8c91af9923ad356eb4448ac2ed15167a9/src/items.rs#L147

There is a minor change in behavior for when multiple methods are chained together - the width used in the error will be the last successfully modified width. I think this is okay?

@camsteffen - re: this item, i've not looked at the diff yet but if there's a change to the formatting output (even if it's an improvement) then we'll need to edition gate it behind style edition 2027

There should be no change in user-facing behavior, formatting. Just a change to the contents of the error object, which is never used IIUC.

authors: calebcartwright camsteffen ding-young


text:
Evaluate collaboration survey length

The interviews took 20~30 minutes, ideally the survey should be much faster.

related to https://github.com/rust-lang/surveys/pull/45

repo: rust-lang/surveys


text:
Enable support for executing arbitrary functions in real mode

Sometimes it is necessary to be able to execute code in real mode, especially if you want / need to use a BIOS function that is only available in real mode. (e.g. the APM Installation check function (God why did I even decide that I needed to support this piece of legacy junk)).
Thats why I propose to add a new field to the BootloaderConfig struct in the bootloader-api, wich takes a FnOnce, which in return is executed some time before the switch to protected mode happens.

This Post in osdev claims that V86 mode is nonexisted when in 64 bit mode (which i think is long mode?)

Right. By the way why are you using an APM function with x86_64? Don't all x86_64 systems support ACPI? And if you can't interact with APM after booting anyway, why do you need to interact with it during booting?

By the way why are you using an APM function with x86_64?

I really just wanted a way to shutdown the system even if there was no acpi table present or there was any other reason why i couldnt interact with the acpi

Don't all x86_64 systems support ACPI?

I really hope this is true, but i cant find anything that prooves or disprooves this statement

And if you can't interact with APM after booting anyway, why do you need to interact with it during booting?

Well, I CAN interact with the apm after booting, since it has a protected mode 32 bit interface (of which im just noticing that it mignt not exactly be available when compiling for x86_64). The installation check however is required since

Newer hardware doesnt support the APM anymore, and
the Installation check is the only way to find out the version and flags of the installed APM
And of course, the Installation check is only available when in real mode

Hmmmm
Considering acpi is a thing since 1996 I think its rather safe to assume that the table is existent in ram and usable...

I'll close this for now. Thank you a lot for your help!

repo: rust-osdev/bootloader


text:
UntaggedEnum with bool variant breaks unexpectedly

This PR does not have a fix yet, but opening for a discussion
  1) test untagged enum transcoder (RustlerTest.CodegenTest)
     test/codegen_test.exs:44
     Assertion with == failed
     code:  assert :invalid_variant == RustlerTest.untagged_enum_echo([1, 2, 3, 4])
     left:  :invalid_variant
     right: true
     stacktrace:
       test/codegen_test.exs:48: (test)

Closing in favor of #219

repo: rusterlium/rustler


text:
Aabb3.transform(Point3). can't be added to scope. Rust wants to use Aabb's.

impl Object {
    pub fn update_collision(&mut self) {
        self.collsion_aabb3.transform(&self.position);
    }
}

I have use collision::Aabb3;
then the rust compiler is telling me to use use collision::Aabb;, but then it doesn't work at all.
everything is f32 aswell.

I miss read the documentation

repo: rustgd/collision-rs


text:
feat: adds support for ignores

Copy of https://github.com/actions-rs/audit-check/pull/221 (and https://github.com/GothenburgBitFactory/audit-check/pull/1!) with one additional commit (dc56dc8ce62eaa70a57ae27c1f61c3221e3299c8).
This isn't strictly necessary -- the action will already follow  .cargo/audit.toml -- but as @pinkforest said elsewhere, sometimes being able to control things in an action separately from in-repo files is useful.

btw - we need to update README for that - wanna send a PR ? cheers

authors: djmitche pinkforest

repo: rustsec/audit-check

license: MIT


text:
Import JS function provided at runtime

Motivation
I'm embedding a Bevy game in a Vue app. i'd like for them to be able to communicate. since there's no apparent way for a rust function called from JS to interact with the ECS (or, i haven't found it) i'd like to be able to call a JS function from rust which returns a queue of recent events.
The function I want to call will be in a .vue file (and, even if i put it in a .ts file, i can't figure out how to get #[wasm_bindgen(module=...)] to play nice with Vite) which makes it impossible to import. So i'd like to be able to provide it at runtime, presumably while calling __wbg_init
as a bonus, this would make it always possible to link to a function with any bundler.
Proposed Solution
#[wasm_bindgen]
extern "C" {
    #[wasm_bindgen(dynamic)]
    fn foo() -> u32;
}

import init, {main} from "my-wasm-whatever";

function whatever() {
    return 1337;
}

init(undefined, {foo: whatever}).then(main);

modify __wbg_init to something like
async function __wbg_init(input, import_overrides = undefined) {
    if (wasm !== undefined) return wasm;

    if (typeof input === 'undefined') {
        input = "/assets/my_wasm_whatever.wasm"
    }
    const imports = __wbg_get_imports();
    if (import_overrides !== undefined){              //   \
        Object.assign(imports, import_overrides);     //    )- the added part
    }                                                 //   /

    if (typeof input === 'string' || (typeof Request === 'function' && input instanceof Request) || (typeof URL === 'function' && input instanceof URL)) {
        input = fetch(input);
    }

    __wbg_init_memory(imports);

    const { instance, module } = await __wbg_load(await input, imports);

    return __wbg_finalize_init(instance, module);
}

Alternatives
If this is already possible i couldn't find it in the documentation.

The documents I found similar to your needs are here, and the relevant code is  here.

It's my idea. declare foo function in #[wasm_bindgen] extern, wasm_bindgen expose a function for import foo function (like set_foo_ns):
#[wasm_bindgen]
extern "C" {
    pub type NS;

    #[wasm_bindgen(method)]
    pub fn foo(this: &NS) -> u32;
}

#[wasm_bindgen]
pub fn set_foo_ns(ns: &NS) -> u32 {
    ns.foo()
}

import wasmInit, { set_foo_ns } from '../pkg';

wasmInit().then( ()=> { console.log(set_foo_ns( { foo: () => 1 } )); } );

It will print 1 on console.

An alternative would be to define a wrapper that lives in a .ts/.js file and calls the Vue function itself, and then import the wrapper into WASM.

I believe the simplest way is

Create a Javascript object
Create a Rust shim for it, along the lines of https://rustwasm.github.io/wasm-bindgen/examples/import-js.html
Add a function that lets you pass in the Javascript object, and store a reference to it. You'll probably need myJsValue.unchecked_into::<MyClass>();
Use it

@stefnotch I think this is a good way. I'm just commenting an example I created when adding dynamic import to another project. (as it wasn't exactly obvious to me on how to fully complete it)
#[wasm_bindgen]
extern "C" {
    #[wasm_bindgen]
    pub type HelloWorldModule;

    #[wasm_bindgen(method, js_name = "helloWorld")]
    pub fn hello_world(this: &HelloWorldModule);

}

pub async fn load_dynamic_hello_world() -> HelloWorldModule {
    let module_as_str = r#"export function helloWorld() { console.log("hello world!"); }"#;

    let from_data = Array::new();
    from_data.push(&module_as_str.into());

    let mut type_set: BlobPropertyBag = BlobPropertyBag::new();
    type_set.type_("application/javascript");

    let blob = Blob::new_with_str_sequence_and_options(&from_data, &type_set).unwrap();
    let module_address = web_sys::Url::create_object_url_with_blob(&blob).unwrap();

    let module_promise: Promise = js_sys::eval(&format!(r#"import ("{}")"#, module_address))
        .unwrap()
        .into();

    let module = JsFuture::from(module_promise).await.unwrap();

    let as_hello_world: HelloWorldModule = module.into();
    as_hello_world.hello_world();

    as_hello_world
}

I like it better than inline_js=, as I think it's likely this will work more frequently with different rust frameworks. I think this because inline_js bindings require the creator of the framework to explicitly support them.

repo: rustwasm/wasm-bindgen


text:
--out flag that allows configuration of ./pkg dir

This is a feature request; I would like to configure the output directory (currently hardcoded to ./pkg).
When Webpack invokes wasm-pack in the root of the project it will create the ./pkg directory. Unlike the normal wasm-pack's use case we won't publish it and don't .gitignore it.
I would like to avoid generating the intermediate files in the user's repository but rather in a temporary fs, where Webpack will be able to bundle it anyway.

I wouldn't mind taking this one! I'll start looking into it, and will follow up if I have any other questions :)

go for it @data-pup that'd be awesome!

In #263, I'm pretty close to a complete solution. What should be printed here? I could imagine writing the absolute path, or trying to use a nicer looking relative path.

I could imagine writing the absolute path, or trying to use a nicer looking relative path.

Either one sounds good to me.

Fixed in #263

authors: ashleygwilliams data-pup fitzgen xmclark xtuc

repo: rustwasm/wasm-pack


text:
Web server code reaching unreachable state.

🐛 Bug description
I try to run some fairly basic code using the built-in std::net library, but then I get this an error:
RuntimeError: unreachable
at __rust_start_panic (wasm://wasm/000cba06:0:134925)
at rust_panic (wasm://wasm/000cba06:0:130833)
at std::panicking::rust_panic_with_hook::h4e1267e42c34e062 (wasm://wasm/000cba06:0:79828)
at std::panicking::begin_panic_handler::{{closure}}::h55e6bb589840bdac (wasm://wasm/000cba06:0:92697)
at std::sys_common::backtrace::__rust_end_short_backtrace::h404ad66b7b407ddd (wasm://wasm/000cba06:0:129000)
at rust_begin_unwind (wasm://wasm/000cba06:0:124973)
at core::panicking::panic_fmt::hfa15f5472ef5e557 (wasm://wasm/000cba06:0:129100)
at core::option::expect_none_failed::h17eb63fc53350311 (wasm://wasm/000cba06:0:97636)
at core::result::Result<T,E>::unwrap::h4554bfc1a1a7c094 (wasm://wasm/000cba06:0:91945)
at wasm_server::serve::{{closure}}::h4f740af078c11550 (wasm://wasm/000cba06:0:11631)
This does not occur when I run it out of a WASM environment.
🤔 Expected Behavior
The code should've listened on port 8080 for incoming connections and logged whenever a connection was established. (Not supposed to send a response, tho!) I was following this tutorial, and the code is almost an exact replica of what is in the first code block.
👟 Steps to reproduce
Step #1: Follow tutorial
Step #2: Attempt to compile this exact code:
mod utils;

use wasm_bindgen::prelude::*;
use std::net::TcpListener;

// When the `wee_alloc` feature is enabled, use `wee_alloc` as the global
// allocator.
#[cfg(feature = "wee_alloc")]
#[global_allocator]
static ALLOC: wee_alloc::WeeAlloc = wee_alloc::WeeAlloc::INIT;

#[wasm_bindgen]
extern {
    #[wasm_bindgen(js_namespace = console)]
    fn log(s: &str);
}

#[wasm_bindgen]
pub async fn serve(port: u16) {
    let listener = TcpListener::bind(&format!("127.0.0.1:{}", port)).unwrap();

    for _stream in listener.incoming() {
        log("Connected!");
    }

    log(&format!("Listening in on: {}", port));
}

Step #3: panic!
🌍 Your environment
Include the relevant details of your environment.
wasm-pack version: wasm-pack 0.9.1 (via wasm-pack -V)
rustc version: rustc 1.53.0-nightly (07e0e2ec2 2021-03-24) (via rustc -V, behavior is the same on the stable toolchain, I've tried.)
Edit: I messed up formatting on initial post. Oops!

I feel like an idiot. I spotted a typo in the code. I am so sorry.

Accidentally closed this! Whoops!


text:
Questions and help on node to node modeling

❓ Questions & Help on node to node modeling
Hi, first thanks for this awesome package, it's a great piece of work.
I've been working on the Molecular Properties challenge on Kaggle lately (https://www.kaggle.com/c/champs-scalar-coupling) and I am struggling a bit. I would really appreciate some help (both in terms of existing codes, papers or your own vision on the problem).
Problem setting
In this competition, we have molecule similar to the molecules in QM9 datasets, the goal is to predict the coupling constant between some atom pairs given atom types and positions. There are 8 differents pair types ('1JHC', '2JHH', '1JHN', '2JHN', '2JHC', '3JHH', '3JHC', '3JHN') and it seems that making different models for each type improve performances.
Here are some specificity I see here and didn't managed to overcome yet :

the edges are not given, so there are different ways of creating the edges. I'll be more specific when explaining my approaches so far.
for a same graph you have multiple targets that don't follow a clear pattern (not one output per node, not one output per edge etc...)

Different approaches
"Real" bond graphs
My first approach consists in creating both node_features, edges and edges_attr by using openbabel and following existing bonds.
Since there is no "real" bond for 3JHC coupling type for example, this method does not allow  to create predictions from edges so  my approach is that one prediction can be done from one graph and two nodes indices where I just stack nodes_1 and nodes_2 representation to then pass then to a MLP (with other external features).
 class RealBondNet(torch.nn.Module):
    def __init__(self, num_node_features, num_edge_features, num_mol_features,
                 dropout=0.0, output_dim=1, num_conv = 5, conv_size=64, num_steps=3):
        super(RealBondNet, self).__init__()
        self.num_conv = num_conv
        self.num_steps = num_steps
        self.num_mol_features = num_mol_features
        self.conv_size = conv_size
        self.output_dim = output_dim
        self.dropout = dropout
        self.graph_conv1 = GraphConv(num_node_features, self.conv_size)
        self.graph_conv2 = GraphConv(self.conv_size, self.conv_size)
        self.pool_S2S = Set2Set(self.conv_size, processing_steps=self.num_steps)

        self.edge_lin0 = torch.nn.Linear(num_node_features, conv_size)

        nn = torch.nn.Sequential(Linear(num_edge_features, 128), ReLU(), Linear(128, conv_size * conv_size))
        self.NNconv = NNConv(conv_size, conv_size, nn, aggr='mean', root_weight=False)
        self.gru = torch.nn.GRU(conv_size, conv_size)
        self.edge_set2set = Set2Set(conv_size, processing_steps=self.num_steps)

        self.fc1 = Linear(self.conv_size*8, self.conv_size*4)
        self.bn1 = BatchNorm1d(self.conv_size*4)
        self.fc2 = Linear(self.conv_size*4, int(self.conv_size))
        self.bn2 = BatchNorm1d(int(self.conv_size))

        # mol features
        self.fcmol1 = Linear(self.num_mol_features, self.num_mol_features)
        self.bnmol1 = BatchNorm1d(self.num_mol_features)
        self.fcmol2 = Linear(self.num_mol_features, self.num_mol_features)
        self.bnmol2 = BatchNorm1d(self.num_mol_features)
        self.fcmol3 = Linear(int(self.num_mol_features), int(self.num_mol_features /2))
        self.bnmol3 = BatchNorm1d(int(self.num_mol_features /2))

        self.before_final_fc = Linear(int(self.num_mol_features /2)+self.conv_size, self.conv_size)
        self.bnbf = BatchNorm1d(self.conv_size)
        self.final_fc = Linear(self.conv_size, 1)

    def forward(self, data):
        x, edge_index, edge_attr, batch, atom_index_0, atom_index_1 = data.x, data.edge_index, data.edge_attr, data.batch, data.atom_index_0, data.atom_index_1

        # edges transformations
        out = F.relu(self.edge_lin0(data.x))
        h = out.unsqueeze(0)

        for i in range(3):
            m = F.relu(self.NNconv(out, edge_index, edge_attr))
            out, h = self.gru(m.unsqueeze(0), h)
            out = out.squeeze(0)

        edge_pool = self.edge_set2set(out, batch)

        # mol information
        mol_out = self.fcmol1(data.mol_attr.view(atom_index_0.shape[0], -1))
        mol_out =  F.relu(self.bnmol1(mol_out))
        mol_out = self.fcmol2(mol_out)
        mol_out =  F.relu(self.bnmol2(mol_out))
        mol_out = self.fcmol3(mol_out)
        mol_out =  F.relu(self.bnmol3(mol_out))
        mol_out = F.dropout(mol_out, p=self.dropout, training=self.training)
        # nodes transformations
        x = F.relu(self.graph_conv1(x, edge_index))
        for i in range(self.num_conv):
            x = F.relu(self.graph_conv2(x, edge_index))

        pool = self.pool_S2S(x, batch)

        nodes_0 = x[atom_index_0,:]
        nodes_1 = x[atom_index_1,:]

        nodes_edge_pool_0 = out[atom_index_0,:]
        nodes_edge_pool_1 = out[atom_index_1,:]

        x = torch.cat([edge_pool, pool, nodes_0, nodes_1, nodes_edge_pool_0, nodes_edge_pool_1], dim=1)
        # BEFORE MLP
        x = self.fc1(x)

        x = F.relu(self.bn1(x))
        x = self.fc2(x)
        x = F.relu(self.bn2(x))
        x = F.dropout(x, p=self.dropout, training=self.training)
        x = torch.cat([x, mol_out], dim=1)

        x = self.before_final_fc(x)
        x = F.relu(self.bnbf(x))
        x = self.final_fc(x)
        #x = self.fc3(x)
        return x

This achieves reasonable but non competitive scores around 0.4 mae while 0.1 is achievable.
I feel like cherry picking nodes representation i and j could break the "flow" since it's external variables which determines what part of the graph representation goes into the MLP.
Fully connected Graphs
An other approach I tried is to systematically creates all edges (n(n-1)/2 edges where n is the number of atoms) and try to make all predictions for all atom pairs at one time using a mask to perform gradient descent only on given connections.
However I had issue with creating my torch geometric dataset because the default collate function does not seem to allow matrix as targets.
I get this error when trying to set data(y=a_matrix) :
---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-7-01bfcf89c013> in <module>
----> 1 dataset = MolecularCouplingDataset()

<ipython-input-6-080f091cda03> in __init__(self, root, transform, pre_transform)
      1 class MolecularCouplingDataset(InMemoryDataset):
      2     def __init__(self, root="/home/seb/MolecularCouplingPredictions/output", transform=None, pre_transform=None):
----> 3         super(MolecularCouplingDataset, self).__init__(root, transform, pre_transform)
      4         self.data, self.slices = torch.load(self.processed_paths[0])
      5

/opt/conda/lib/python3.6/site-packages/torch_geometric/data/in_memory_dataset.py in __init__(self, root, transform, pre_transform, pre_filter)
     54                  pre_filter=None):
     55         super(InMemoryDataset, self).__init__(root, transform, pre_transform,
---> 56                                               pre_filter)
     57         self.data, self.slices = None, None
     58

/opt/conda/lib/python3.6/site-packages/torch_geometric/data/dataset.py in __init__(self, root, transform, pre_transform, pre_filter)
     81
     82         self._download()
---> 83         self._process()
     84
     85     @property

/opt/conda/lib/python3.6/site-packages/torch_geometric/data/dataset.py in _process(self)
    125
    126         makedirs(self.processed_dir)
--> 127         self.process()
    128
    129         print('Done!')

<ipython-input-6-080f091cda03> in process(self)
     33             data_list.append(data)
     34
---> 35         data, slices = self.collate(data_list)
     36         torch.save((data, slices), self.processed_paths[0])
     37

/opt/conda/lib/python3.6/site-packages/torch_geometric/data/in_memory_dataset.py in collate(self, data_list)
    148             if torch.is_tensor(data_list[0][key]):
    149                 data[key] = torch.cat(
--> 150                     data[key], dim=data.__cat_dim__(key, data_list[0][key]))
    151             else:
    152                 data[key] = torch.tensor(data[key])

RuntimeError: invalid argument 0: Sizes of tensors must match except in dimension 0. Got 5 and 4 in dimension 1 at /opt/conda/conda-bld/pytorch_1556653099582/work/aten/src/TH/generic/THTensor.cpp:711

I tried to overcome this by flattening everything but I struggled with reshaping everything back during  the forward step.
Anyway I decided to create create fake nodes to pad every graph to the maximum size and  I tried to apply a MPNN, then from a matrix (n_graphs, n_conv, n_atom) I tried to create a invariant matrix of shape  (n_graphs, n_conv, n_atom, n_atom) to then apply 1x1 conv2D to get predictions for all edges. Here is the network, results where pretty bad but I am not sure I managed to handle the batches properly.
class SimpleNet(torch.nn.Module):
    def __init__(self, num_node_features, num_edge_features,
                 dropout=0.0, output_dim=1, num_conv = 6, conv_size=64, num_steps=8):
        super(SimpleNet, self).__init__()
        self.num_conv = num_conv
        self.num_steps = num_steps
        self.conv_size = conv_size
        self.output_dim = output_dim
        self.dropout = dropout

        self.edge_lin0 = torch.nn.Linear(num_node_features, conv_size)

        nn = torch.nn.Sequential(Linear(num_edge_features, 128), ReLU(), Linear(128, conv_size * conv_size))
        self.NNconv = NNConv(conv_size, conv_size, nn, aggr='mean', root_weight=False)
        self.gru = torch.nn.GRU(conv_size, conv_size)

        self.conv2d =torch.nn.Conv2d(self.conv_size, 1, kernel_size=1)

    def forward(self, data):
        x, edge_index, edge_attr, batch = data.x, data.edge_index, data.edge_attr, data.batch

        out = F.relu(self.edge_lin0(data.x))
        h = out.unsqueeze(0)

        for i in range(self.num_steps):
            m = F.relu(self.NNconv(out, edge_index, edge_attr))
            out, h = self.gru(m.unsqueeze(0), h)
            out = out.squeeze(0)

        reshaped_out = out.view(data.num_graphs, 29, self.conv_size).permute(0, 2, 1)
        order_invariant_m = reshaped_out.reshape(data.num_graphs, self.conv_size, 29, 1) @reshaped_out.reshape(data.num_graphs, self.conv_size, 1, 29)

        res_conv2D = self.conv2d(order_invariant_m)

        return res_conv2D

So a few questions:
1- do you know about such problems that have been tackled before? Do you have any links?
2- how would you approach the problem yourself?
3- which method looks more promising to you?
4- any idea how to overcome the collate error and simply put matrices as targets?
5- does it make any sense to pad every graph to the maximum graph size? how to do so properly?
6- any idea how to deal with training one or different model to take into account the different coupling types?
I hope this message finds you well (I get that it's a bit hairy) but I would really be glad to have your thoughts on this.
Thanks!

Hi and sorry for answering late, but this is an in-depth question and I hasn‘t habe much time recently. To be honest, I haven‘t looked into the Kaggle competition that closely and I am far from expert on this field, so please take my advice with caution. Overall, I find the best place that describes the problem from a GNN perspective is this thread. IMO, I would tackle this problem similar.
1, 2 and 3: While trying to replicate the results of Gilmer et al, I noticed that it is extremely important to perform graph convolution on the fully-connected graphs with distances and bond types as edge features, so I do believe this is the way to go.
4. This is an extremely interesting problem and you may overcome it by saving sparse target matrices in analogy to edge_index and edge_attr, e.g., y_index and y_value. Batching of those values should work fine. Alternatively, dense matrices should work fine too, but they all need to be of equal size. You can save them as [1, N, N] shape to ensure correct batching.
5. This does make sense when you want to model pair-wise interactions. We provide such functionality with to_dense_batch and to_dense_adj in torch_geometric.utils.
6. Do you want to train a single model for all types? I guess you will need to create targets for each coupling type (in sparse scenario) or with an additional dimension (in dense scenario).

Thanks for your answer, it helped me a lot.
I implemented a fully connected version using n(n-1)/2 edges, with corresponding mask and targets for node to node predictions. Putting only half of the edges since my graph is undirected saves space and time. However I am trying to apply a stack of MetaLayers that seem to be working but not as good as I'd expect. I am wondering if this could be due to using only half edges.
In the original paper, the graphs are directed and so information is aggregated following directed eges. My node model looks like this
class NodeModel(torch.nn.Module):
    def __init__(self, num_node_features, num_u_features, node_conv_dim, edge_model_dim):
        super(NodeModel, self).__init__()
        node_mlp1_dim = node_conv_dim
        in_mlp_1 = num_node_features + edge_model_dim
        in_mlp_2 = node_mlp1_dim + num_node_features + num_u_features
        self.node_mlp_1 = Seq(Lin(in_mlp_1, node_mlp1_dim), ReLU(), Lin(node_mlp1_dim, node_mlp1_dim), BatchNorm1d(node_mlp1_dim))
        self.node_mlp_2 = Seq(Lin(in_mlp_2, node_conv_dim), ReLU(), Lin(node_conv_dim, node_conv_dim), BatchNorm1d(node_conv_dim) )

    def forward(self, x, edge_index, edge_attr, u, batch):
        # x: [N, F_x], where N is the number of nodes.
        # edge_index: [2, E] with max entry N - 1.
        # edge_attr: [E, F_e]
        # u: [B, F_u]
        # batch: [N] with max entry B - 1.
        row, col = edge_index
        out = torch.cat([x[col], edge_attr], dim=1)
        out = self.node_mlp_1(out)
        out = scatter_mean(out, row, dim=0, dim_size=x.size(0))
        out = torch.cat([x, out, u[batch]], dim=1)
        return self.node_mlp_2(out)

But this part out = torch.cat([x[col], edge_attr], dim=1)         out = self.node_mlp_1(out)         out = scatter_mean(out, row, dim=0, dim_size=x.size(0)) makes me wonder, am I missing half of the messages compared to a fully connected n(n-1) version? It seems that my message fly only from smaller nodes index to larger nodes index (because i have edge_index looking like this [[1, 2, 2, 3, 3, 3], [0, 0, 1, 0, 1, 2]]). Isn't that a big problem in my model?
If so I see two ways to overcome this :

double the number of edge_index to have all directions

try to change my NodeModel to pass messages from both ways.
Which one would you recommend? How would you implement 2)?

Hope my questions make sense, thanks a lot!

Yes, you only propagate along half of your edges. I recommend doubling the number of edges. This way you may have more GPU utilization. But honestly, you do not need scatter operations for fully connected graphs. Currently, you implement a sparse version for a very dense graph, which is not that useful. A dense version of your approach should be way faster.

Ok thanks, I doubled the edges and it worked better! Twice longer thoug...
Can I easily switch to a dense implementation with TorchGeometric?

You can, via the ToDense transform and the DenseDataLoader, but we only provide s limited number of dense GNN operators, although it should be straightforward to extend them or build your own.


text:
Atmospheric pressure display needs option for inHg (Enhancement)

As with temperature,
Atmospheric pressure should have settings to display in hPa, in and mm (mbar?)
The legend need not include Hg.
For example 992.67hPa display as 29.31

inHg and mmHg added as pressure units


text:
Default args in components?

Creating a font awesome component in Solid.  Would like to have defaults to you don't have to set for example the base every time and show a blatant default name prop to make sure its temporary.
import { IconName, IconPrefix } from '@fortawesome/fontawesome-common-types'
import { classes } from 'typestyle'
import { setDefaults } from 'solid-js'

export interface ButtonProps {
  name: IconName
  base: IconPrefix
}

export default function FontAwesomeIcon(props: ButtonProps) {
  setDefaults(props, { name: `biohazard`, base: `fas` })
  const iconClasses = classes(props.base, `fa-${props.name}`)
  return <i class={iconClasses} />
}

Trying to use it without all props ala <FontAwesomeIcon /> gives
Type '{}' is not assignable to type 'IntrinsicAttributes & ButtonProps'.
  Type '{}' is missing the following properties from type 'ButtonProps': name, base

Found the setDefaults function but doesn't appear to work like I'd expect.  What am I doing wrong?

I think that if your props are optional then you need to indicate with the props definition. Admittedly I haven't done enough TypeScript development with other libraries but it makes sense to me to work this way since the Component definition should be able to define what is required.
What happens if you change it to (notice the question marks)?
interface ButtonProps {
  name?: IconName
  base?: IconPrefix
}

100% correct, getting used to JSX... thanks.

Yeah.. JSX + TypeScript might be about the worst part of TypeScript. It was my introduction unfortunately, I came across(existing) and wrote about 5 issues just in the first week or so of learning TypeScript. Not really the best introduction.

authors: delaneyj ryansolid

repo: ryansolid/solid


text:
Inheritance Method Names

I'm debating the semantics of blood's method names. I'd like the names to be intuitive and terse. The 2 inheritance methods are opposites of each other:
version 0.1 method names:

blood.child(parent) is like Object.create(parent)
blood.parent(child) is like Object.getPrototypeOf(child)

alternative name ideas:

blood.create(parent) or blood.nu(parent)
blood.pro(child) or blood.proto(child)

What names should we use?

aliasing .child and .create for now

repo: ryanve/blood


text:
🛑 NC Gazetteer is down

In cf15649, NC Gazetteer (https://ncgazetteer.org) was down:

HTTP code: 0
Response time: 0 ms

Resolved: NC Gazetteer is back up in 4b57762 after 6 minutes.

repo: rybesh/status


text:
video running very slow

videotest.py implemented but the video runs very slow at almost 1-FPS. Suggest some solution

@s-sourav15 Hey have you found any solution? I tried to implement multi threading, but it runs into some error.

license: mit


text:
Support Jupyter Notebook(.ipynb)

Is your feature request related to a problem? Please describe.
Currently, only .md files are supported. Jupyter Notebooks(.ipynb) are not supported.
Describe the solution you'd like
A .ipynb file is a json file containing code cells and markdown cells. At present, applying the parsing rules to markdown cells could provide this feature.
Describe alternatives you've considered
Decomposing the logic of MultilingualDoc and provide a convert API would also help. Imagine the API convert:
# raw markdown text
raw_markdown:str = open(path_to_md)

# converted texts
converted_markdowns: List[str] = convert(raw_markdown)

for txt in converted_markdowns:
    save(txt)

If this API is provided, it would be convenient for others to provide support to other file format
Additional context
It would be better if the convert could apply to ipynb's code part.

Thank you for your great suggestion.
1. Jupyter Notebook support (.ipynb) I hadn't considered Jupyter Notebooks at all, but they look useful. In the next coming version, I will add a feature for the cells of "cell_type": "markdown" in .ipynb files.
2. Python API for the mmg I agree with you that we need a simple API. I'll add it in the next version.
3. Code cell support in Jupyter I'm sorry, I didn't understand how you want to use the converting function on Code cells. Could you give me examples or describe it in more detail?

Thank you for your reply.  For the code cell part, I mean there are often situations comments are needed to explain the code in detail. And it would be better if the multilingual feature could apply to the comments. The following code example explains the situation that I might want to translate "initialize the settings" to other languages
# initialize the settings
init()

Thanks, now I understand. :) 👍
It would be great if people could use mmg in code cells. However, we need some discussion to define the proper usage rules because various complex situations are possible.
Since the code cell will call a program logic, the section keyword must be written as a comment. And a comment can also be on the same line as the code. Furthermore, I've seen someone use other languages like C++ in Jupyter.
It is possible to put restrictions on the user environments, but this may increase the complexity of user understanding.


text:
[BUG] @Singular does not work properly with @Jacksonized

Description
If a field will be annotated with @Singular and @JsonProperty("someOtherName") and the class also has @Jacksonized with @(Super)Builder, Jackson cannot normally deserialize this field.
Possible behaviors with @Singular#value and default ObjectMapper:

Empty value - "Unrecognized field someOtherName";
Value equals to field name - "Conflicting setter definitions for property fieldName";
Value equals to @JsonProperty#value - "out of START_ARRAY token" - normal behavior;

To Reproduce
Create any POJO annotated with @jacksonized and @(Super)Builder and field with @Singular and @JsonProperty("otherName") and try to deserialize object from string with Jackson.

Lombok EDGE, latest AdoptOpenJDK-14.0.1.7

The current edge was built on 2020-03-15, PR #2429 (where @Singular support was implemented) was merged 2020-04-17. So it's not in edge, yet.
You could build from source, if you want to test it.

@janrieke, Ok, thanks. I will close this issue.

It's now in the edge release. Could you verify that it works, or otherwise reopen this?

@janrieke, yes, i think everything good with @Singular.

Thanks for the feedback

authors: Bittuw janrieke rspilker

repo: rzwitserloot/lombok


text:
Cannont exit terminal

I am using Neovim 0.7.2 and have installed Neovim-Terminal using Packer. I can open the terminal using  + 1 or any other method. I can also toggle it. However, when I move to insert mode, I cannot switch between panes or even press [ESC]. I have to execute exit to close the terminal. I use Bash for my terminal. Please help me.

https://vi.stackexchange.com/a/4922


text:
Add JWT authentication

Hello there !
I imagine that a lot of API's resulting from this repository ends up having an authentication system.
I was thinking it may be a good idea to put an option to generate some kind of boilerplate for JWT authentication since, I assume most of people used nowadays.
What do you think about it ?

Yeah. That's a great point.
For this issue I guess the best way is to implement it by hand. Because FastAPI-JWT isn't maintained now. Or it's a good opportunity to write fastapi-jwt library from scratch.

Okay, so this implies that we'd need to create a /signup  and a /login route. But also that we provide an User model right ?
I don't know how you see the JWT mechanism, to me it would be simple : the /login route generates and returns a JWT token that expires after some time, we use it as access token.
I don't know if we want to do all the authorization_code, refresh token, access_token  "dance".
And maybe we could add an example of a protected route.
What do you think ?

I can give it a go and set it up with Ormar which I used recently, but I guess I'll need help for the other ORMs.

How can JWT be related to ORMs?
Maybe it would be nice to add the JWT service. It may not depend on any ORM. You will add methods to encode and decode tokens. The rest is up to people.
Because one time, I needed JWT authentication for a service without a database.

But of course, suggestions and Pull requests are welcome.

Is this currently being worked on or is open to be implemented?

Currently it's open to contributions. Personally I would prefer to create a library that can handle different JWT scenarios.
After the lib is ready, it's easy to add it in the template.
But if you have another solution in mind, feel free to open pull request with your changes.

@s3rius maybe it is possible to use https://github.com/fastapi-users/fastapi-users ? They provide more or less ready-made solution for auth, supporting JWT and other methods. I think it is very widely used, although they have removed support for ORMs besides SQLAlchemy, but I am currently writing an extension for ormar and tortoise.

Hi @jegork and thanks for advise. I don't have much experience with fastapi-users, but It would be really nice if this thing can cover all user-related questions.

@s3rius Hi, I needed something like this as weel and I have tried to implement the samples from fastapi-users and it seems to work fine out of the box. However I have never worked with creating cookiecutters themselves, so idk if I may have missed something major or there is something more I should include before opening a PR.
https://github.com/s3rius/FastAPI-template/compare/master...andcarnivorous:FastAPI-template:master

@andcarnivorous Hi. Thanks for your contribution! Just open the pr, and we'll discuss everything within it.

can you give me the DATABASE template

Run the cli with python3 -m fastapi_template

Give the project name and choose:

REST API
SQLITE/Postgres database
SQLAlchemy as ORM
Add fastapi-users support in the section.additional tweaks

test http://127.0.0.1:8000/api/docs#/auth/register_register_api_auth_register_post.
it will return Internal Server Error , because

2023-09-25 00:56:54.768 UTC [473] STATEMENT:  SELECT "user".id, "user".email, "user".hashed_password, "user".is_active, "user".is_superuser, "user".is_verified
FROM "user"
WHERE lower("user".email) = lower($1::VARCHAR)

2023-09-25 01:01:47.315 UTC [473] ERROR:  relation "user" does not exist at character 121

maybe i need the user DATABASE template, can you help me?
@andcarnivorous @s3rius

Hi, @vmjcv. The table should follow this protocol: https://github.com/fastapi-users/fastapi-users/blob/ff9fae631cdae00ebc15f051e54728b3c8d11420/fastapi_users/models.py#L6. Anyway, I consider this as a problem, for sure. We need to fix that.
Thanks noticing.

Hey, are you running the application with docker compose? If not, how are you running the application? @vmjcv

Hey, are you running the application with docker compose? If not, how are you running the application? @vmjcv

yes, i use it:
docker-compose -f deploy/docker-compose.yml -f deploy/docker-compose.dev.yml --project-directory . build
docker-compose -f deploy/docker-compose.yml -f deploy/docker-compose.dev.yml --project-directory . up --build
The reason for the problem is that the user table does not exist in the database of the template project

嗨，.该表应遵循以下协议：https://github.com/fastapi-users/fastapi-users/blob/ff9fae631cdae00ebc15f051e54728b3c8d11420/fastapi_users/models.py#L6。无论如何，我认为这是一个问题，当然。我们需要解决这个问题。
谢谢注意到。

Thank you, that's exactly what I need

@s3rius is this something that should be fixed somewhere specifically? I have tried to repro but whenever I create a new project and spin up docker compose, I can create users immediately and they are indeed in the postgres table.

I will take a look today.

According to this code, I called the create_db_and_tables function to solve the problem of no tables.
I didn't setup Alembic because I'm just a newbie and don't know what Alembic is for.
Finally I modified the _setup_db code and register_startup_event code
async def _setup_db(app: FastAPI) -> None:  # pragma: no cover
    """
    Creates connection to the database.

    This function creates SQLAlchemy engine instance,
    session_factory for creating sessions
    and stores them in the application's state property.

    :param app: fastAPI application.
    """
    engine = create_async_engine(str(settings.db_url), echo=settings.db_echo)
    session_factory = async_sessionmaker(
        engine,
        expire_on_commit=False,
    )
    app.state.db_engine = engine
    app.state.db_session_factory = session_factory

    ## Modify code
    async with engine.begin() as conn:
        await conn.run_sync(Base.metadata.create_all)

def register_startup_event(app: FastAPI) -> Callable[[], Awaitable[None]]:  # pragma: no cover
    """
    Actions to run on application startup.

    This function uses fastAPI app to store data
    in the state, such as db_engine.

    :param app: the fastAPI application.
    :return: function that actually performs actions.
    """

    @app.on_event("startup")
    async def _startup() -> None:  # noqa: WPS430
        app.middleware_stack = None
        ## Modify code
        await _setup_db(app)
        init_redis(app)
        init_rabbit(app)
        setup_prometheus(app)
        app.middleware_stack = app.build_middleware_stack()
        pass  # noqa: WPS420

    return _startup

I don't know if any of this will help you guys, I'm just a newbie

Guess that was the problem. Because Alembic should create all tables.
Basically it's a really great thing for creating database migrations. Please consider using it for all new projects. But that thing should be unrelated to the problem.
But anyway, I also encoutered this problem.
meme-api-1       | sqlalchemy.exc.ProgrammingError: (sqlalchemy.dialects.postgresql.asyncpg.ProgrammingError) <class 'asyncpg.exceptions.UndefinedTableError'>: relation "user" does not exist
meme-api-1       | [SQL: SELECT "user".id, "user".email, "user".hashed_password, "user".is_active, "user".is_superuser, "user".is_verified
meme-api-1       | FROM "user"
meme-api-1       | WHERE lower("user".email) = lower($1::VARCHAR)]
meme-api-1       | [parameters: ('user@example.com',)]
meme-api-1       | (Background on this error at: https://sqlalche.me/e/20/f405)

We need to fix that. Please keep this issue open. I'll close it once resolved.

Help
I'm facing same error (user table doesn't exist thingy) above.
Any temporary solutions for this?

@s3rius To address the setup issue, I followed these steps, which may assist in resolving it during your setup process:

Initialization of Docker Environment: Executed the standard Docker Compose command:
docker-compose -f deploy/docker-compose.yml -f deploy/docker-compose.dev.yml --project-directory . up --build

This initializes the Docker environment with the necessary configurations and builds the project.

Database Migration Preparation: Inside the Docker API shell, I executed the Alembic command to autogenerate a new revision:
alembic revision --autogenerate -m "create user table"

This step creates a new database migration script for the user table.

Dependency Import: In the newly generated migration file, I added an import statement:
import fastapi_users_db_sqlalchemy

This resolves the fastapi_users_db_sqlalchemy missing error by explicitly importing the required module.

Database Migration Execution: Back in the Docker API shell, I executed the Alembic upgrade:
alembic upgrade head

This applies the latest migration to the database, effectively creating the user table.

Dependency Addition in pyproject: Modified the pyproject.toml file, specifically under [tool.poetry.dev-dependencies], by adding:
bcrypt = "4.0.1"

This addition resolves an issue with the bcrypt package.

Docker Restart: Finally, I stopped the Docker environment and re-ran it to ensure all changes were properly applied and integrated.


text:
Kafka container failed to start ...

Just started a new project with Kafka dependency, got this error when running docker-compose:

docker-compose -f deploy/docker-compose.yml -f deploy/docker-compose.dev.yml --project-directory . up --build

kafka-1          | [2024-06-26 04:23:45,075] INFO Loading logs from log dirs ArrayBuffer(/bitnami/kafka/data) (kafka.log.LogManager)
kafka-1          | [2024-06-26 04:23:45,160] INFO Attempting recovery for all logs in /bitnami/kafka/data since no clean shutdown file was found (kafka.log.LogManager)
kafka-1          | [2024-06-26 04:23:45,514] INFO Loaded 0 logs in 438ms. (kafka.log.LogManager)
kafka-1          | [2024-06-26 04:23:45,535] INFO Starting log cleanup with a period of 300000 ms. (kafka.log.LogManager)
kafka-1          | [2024-06-26 04:23:45,606] INFO Starting log flusher with a default period of 9223372036854775807 ms. (kafka.log.LogManager)
kafka-1          | [2024-06-26 04:23:46,012] INFO Starting the log cleaner (kafka.log.LogCleaner)
kafka-1          | [2024-06-26 04:23:47,366] INFO [kafka-log-cleaner-thread-0]: Starting (kafka.log.LogCleaner)
kafka-1          | [2024-06-26 04:23:59,841] INFO [BrokerToControllerChannelManager broker=1 name=forwarding]: Starting (kafka.server.BrokerToControllerRequestThread)
dependency failed to start: container market_insights-kafka-1 exited (137)
make: *** [dev] Error 1

How to understand this error? I'm running this on my Mac machine. Thanks

using the latest kafka docker image solves the problem:
kafka:
    image: "bitnami/kafka:latest"


text:
bug: Problems with Drawer

What version of daisyUI are you using?
3.0.25
Describe your issue
Hello everyone. I have two problems with Drawer components. First problem is z-index digits from step component bigger than drawer overlay. Second problem is enabling click on Collapse when drawer is visible.
first problem:

second problem:
https://github.com/saadeghi/daisyui/assets/41111309/6458cb70-25c0-4287-85f5-83954f61f684
What browsers are you seeing the problem on?
Google Chrome (114.0.5735.199)
Reproduction URL
No response

Can you provide a code example about the second problem? Because I can't reproduce that (I can't click the items under the overlay)

Wanted to confirm that I'm also observing the same issue with z-index items not being hidden being the overlay -- in this case using a indicator-item class on a badge component.

+1
https://github.com/MilesWuCode/nuxt3-demo

This problem is due to transform css property applied on all drawer-side children. It creates a new stack and z-index no longer works correctly. More info here (https://philipwalton.com/articles/what-no-one-told-you-about-z-index/) and here (https://developer.mozilla.org/en-US/docs/Web/CSS/CSS_Positioning/Understanding_z_index/The_stacking_context)
The specific rule causing that is
.lg\:drawer-open > .drawer-toggle ~ .drawer-side > *:not(.drawer-overlay) {
        transform: translateX(0%)}

The workaround would be to explicitly assign a position: relative and a z-index other than empty to the things that are layering unexpectedly.
Unfortunately, in my case, I need an absolute positionning on the element which behaves strangely...

Any updates on this or other fixes besides what @CharlieEtienne mentioned?

This is no longer an issue because that CSS transform rule is removed from daisyUI last year.

authors: CharlieEtienne MilesWuCode dbirak mpearce-bain saadeghi vivere-dally

repo: saadeghi/daisyui


text:
Fix typo

#10

Thanks

@sabesansathananthan need to do any other changes?

repo: sabesansathananthan/React-GitHub-Resume


text:
Disabled invoice view and restirct no. of recipient emails

This changes are build on main branch code and not on develop branch code.
The reason is as the develop branch is not tested yet I'm using main branch code to implement this fix.
I will test this change in staging and deploy it to production.
Screenshot:

Need to refactor the spam-related changes from the user model, especially the spam errors

Sure will do that after deploying the fix to production

authors: gowsik-ragunath prasanthchaduvula

repo: saeloun/miru-web


text:
🛑 Hacker News is down

In ea4d1ea, Hacker News (https://news.ycombinator.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Hacker News is back up in edd29fa after 1 hour, 59 minutes.


text:
witch address

Related Safe Addresses
0x798d328407055649acd4360201912e375413e999
0x05c81b8886a8f7c7efbac200d9effd8d7921200b
0xa4a5971ab41cd5af29d11849ab2a376c90f65441
0x59c97465be5b3c3fe1032ce30dd74404564d20dd
0x9d9d9ea1d9422375c6696c953cfba966e6c6c3a0
0xf0833df8eab2f7c4fd1f2d36831364103e844197
0xdd0a87380f8cd911e4ce2dfa82d8c983b002163d
0x8948837501562c53648a82fc5461942189bc65de
0x556f7b2318ec0060c8adc81982838e1a2e10315c
0xd58ac7c455fa89a64bdb271b1237f9d083f639cc
0x6c1cb30ad73f628fb5d0e99a94648a5802f6b51d
0x33d1f73fd14c73f064e4eb0217d7b56b2e0aef8c
0x7fddbf26e0d413f4b18cc8ecd8cb6cb1830f195b
0x069e1ea1c27d09c569cb8b24c7035fdefeed8e73
0x381033fd735a7b7b25d7c75674913e9a7259de4f
0x261c4a747ead7372a314567ed78c9ef7b9b5f5b9
0x014145161be0a3b0fb9000803d1bdddabe532d0e
0x4837044eb492842d6a504475bcf6bd014fb16fb0
Reasoning
All the above addresses are transferred from 0x5769770F5EFe8fB017FB09B6dE3B2d096668377D to eth, which is obviously witch
Methodology
https://etherscan.io/tx/0x0bda5a5272520cbd9761b7d1ac95e4d4fc516ba3668ce2e4c224feb42706bbcb
https://etherscan.io/tx/0xe762691cd6a72c47a69b7d4f5c25281ba22796642e57871c0683a8f736a4e53a
https://etherscan.io/tx/0x9b6b52fa18aa5c0bab842ac710e9c2effee02831ebb121afb239d158fa1e25bd
https://etherscan.io/tx/0x17f7ce028f3f7eac0ceca1bba386ded9309060fb306ef5f0a74ed66b32083e08
https://etherscan.io/tx/0x17f7ce028f3f7eac0ceca1bba386ded9309060fb306ef5f0a74ed66b32083e08
Safe Address
0x188950d274A7C23Ec2e180Aa0E5cf92997B69D70

Related Safe Addresses
0x798d328407055649acd4360201912e375413e999 0x05c81b8886a8f7c7efbac200d9effd8d7921200b 0xa4a5971ab41cd5af29d11849ab2a376c90f65441 0x59c97465be5b3c3fe1032ce30dd74404564d20dd 0x9d9d9ea1d9422375c6696c953cfba966e6c6c3a0 0xf0833df8eab2f7c4fd1f2d36831364103e844197 0xdd0a87380f8cd911e4ce2dfa82d8c983b002163d 0x8948837501562c53648a82fc5461942189bc65de 0x556f7b2318ec0060c8adc81982838e1a2e10315c 0xd58ac7c455fa89a64bdb271b1237f9d083f639cc 0x6c1cb30ad73f628fb5d0e99a94648a5802f6b51d 0x33d1f73fd14c73f064e4eb0217d7b56b2e0aef8c 0x7fddbf26e0d413f4b18cc8ecd8cb6cb1830f195b 0x069e1ea1c27d09c569cb8b24c7035fdefeed8e73 0x381033fd735a7b7b25d7c75674913e9a7259de4f 0x261c4a747ead7372a314567ed78c9ef7b9b5f5b9 0x014145161be0a3b0fb9000803d1bdddabe532d0e 0x4837044eb492842d6a504475bcf6bd014fb16fb0
Reasoning
All the above addresses are transferred from 0x5769770F5EFe8fB017FB09B6dE3B2d096668377D to eth, which is obviously witch
Methodology
https://etherscan.io/tx/0x0bda5a5272520cbd9761b7d1ac95e4d4fc516ba3668ce2e4c224feb42706bbcb https://etherscan.io/tx/0xe762691cd6a72c47a69b7d4f5c25281ba22796642e57871c0683a8f736a4e53a https://etherscan.io/tx/0x9b6b52fa18aa5c0bab842ac710e9c2effee02831ebb121afb239d158fa1e25bd
Safe Address
0x188950d274A7C23Ec2e180Aa0E5cf92997B69D70

All the above addresses are transferred from 0x5769770F5EFe8fB017FB09B6dE3B2d096668377D to eth, which is obviously witch

Please elaborate on how transferring ETH from 1 account to many Safes is airdrop farming per se.

All the above addresses are transferred from 0x5769770F5EFe8fB017FB09B6dE3B2d096668377D to eth, which is obviously witch

Please elaborate on how transferring ETH from 1 account to many Safes is airdrop farming per se.

0x569770F5EFe8fB017FB09B6dE3B2d096668377D is a personal address. The first transaction of the above safe deposit box address is the eth transferred from 0x5769770F5EFe8fB017FB09B6dE3B2d096668377D. Without the eth safe address transferred to this first transaction, no other on-chain behavior can be generated. This operation shows that the above safe deposit box address belongs to The same user, this is obvious witch behavior, real users will not appear such operations.

We haven't received any further information on this ticket and therefore deemed this submission invalid.
Unfortunately just the pure fact that Safes...

were created by the same address
have the same owner
conducted the same transactions
received assets from the same address

... is not enough reasoning for a valid submission, as there are legit user behaviour leading to such as well (relayer, scripts, power users, exchanges)
This submission didn't provide sufficient indicators to convince the team that there is some targeted airdrop farming behind the provided addresses. As per the rules, when in doubt we will default in favor of the airdrop recipients in order to not risk any false positives when sorting out airdrop hunters.

authors: lukasschor tschubotz xiangdaye


text:
The user's container name can be occupied by others.

A user's all container names are unique but he/she can create container having names used by other user.

@lk668

merged in #59 .Thanks @lk668


text:
Graceful handover of conflicts

Where a merge does not encounter any code motion divergence, but conflicts are detected when sections of code are merged, then Kinetic Merge should write stage entries for Git for each file containing conflicts; and should also write the partially-merged form of the file at the expected path, with the usual three-way conflict markers in place.
The standard MERGE_HEAD ref should be set to refer to branch that the user is merging from, mirroring what git merge would do.
Where code motion causes the two sides of the conflict to originate from separate files, this should be reflected properly in the conflict markers, analogous to how Git represents conflicts caused where on branch has renamed an entire file. The situation is more subtle here, as code motion may simply move a section from one file to another; indeed the file with conflicts may have several conflicts involving several files rom which the conflicting sections were moved from.
Out of scope - conflicts caused by code motion on a section of code on one branch, and outright deletion of the same section in another branch. That is for a following story.

This ended being delivered as part of issue #1, when the application shell was being worked on.

repo: sageserpent-open/kineticMerge


text:
🛑 Vedik Gurukulam is down

In 1ffc432, Vedik Gurukulam (https://vedikgurukulam.org) was down:

HTTP code: 500
Response time: 605 ms

Resolved: Vedik Gurukulam is back up in 88bba53.

repo: sahajananddigital/status


text:
No Cloudflare URL on Google Colab: Exception in thread Thread-12: / 1 required positional argument: 'metrics_port'

Error persists, after the issue was closed.
Cloudflare URL is not generated / reported on Google Colab.
Server still starts, but no front end access without the Cloudflare backend url.

Exception in thread Thread-12:
Traceback (most recent call last):
File "", line 14, in start_cloudflared
TypeError: _run_cloudflared() missing 1 required positional argument: 'metrics_port'

I've reopened the previous issue, let's resolve there instead of having a duplicate.

authors: chemaomao saharmor

repo: saharmor/dalle-playground