text:
RFC: native C-compatible unions via untagged_union

Rendered
Discussion on rust-internals
In this RFC, I've suggested the introduction of a new keyword, untagged_union.  I don't know Rust's formal policy for handling the introduction of new keywords while retaining backwards compatibility.  For the purposes of this RFC, I've assumed a policy along the lines of "don't introduce a keyword that breaks existing code, based on crater runs, code searches, expectations of common usage, and similar".  This keyword (unlike union) seems unlikely to introduce such breakage, and searches don't turn up any uses.
However, if introducing a new keyword proves a sticking point for the RFC, I can easily change the RFC to use a different declaration syntax, while retaining the syntax and semantics for using unions.  I previously had a draft RFC that used the syntax #[repr(union)] struct, which would introduce no new keywords; I switched to a keyword based on IRC discussions, and in particular, a strong objection to using the word struct to declare something with overlapping fields.  Nonetheless, the exact syntax of the declaration does not seem to deeply matter to most people in the discussion, and in particular it seems far less important than the syntax and semantics for using unions.

It might be worth summarising some of the discussion from the internals thread - there is a lot of it and it's not easy to follow the threads of conversation.
I strongly agree that we should support untagged unions in Rust. However, I think that #[repr(union)] enum  is the better way to do it - it is more flexible since the variants can have multiple fields, it saves us adding another data structure, and I think unions fit the enum intuition more closely than structs.
Unions as enums works particularly nicely if are able to use variants as types (which may well be a long way off or may never happen, sadly). In that case only the downcast from the enum type to the variant type has to be unsafe (which it would be for any enum, I imagine) and then other use of the variant can be in safe code. In this case, the only difference for repr(union)/unsafe enums is that you can't match them.

See the mention of unsafe enum in the alternatives section; @retep998 wrote an RFC for that.
I can certainly see the argument for that, given that Rust enums represent tagged unions.  However, modeling untagged unions on enums produces some syntactic challenges, though.  How do you access a field of a union?  Enum normally only supports pattern-matching syntax; since the pattern-matching requires unsafe code, pulling out a field F would require something like this: let f = unsafe { let MyUnion::F(f) = u; f };.  That gets uglier if you have structs inside unions inside structs, which FFI interfaces actually use quite often (a struct of simultaneously valid fields, inside a union of many such structs, inside an outer struct containing a tag and other common fields).  At that point, pattern matching for field access starts to approach the level of syntactic complexity introduced by using macros to define and access union fields; in particular, it does not naturally chain as well, and it requires mixing reading right-to-left with left-to-right ordering.
I suspect such syntax would also drive people to include more code in the unsafe block than necessary.
By contrast, field access syntax would simplify that to let f = unsafe { u.f };  And even with nested structures and unions, you'd have something like let f = unsafe { s.u.f.x };.
As discussed in the rust-internals thread and mentioned in the alternatives section of this RFC, you could potentially support struct field access syntax with unsafe enum, allowing access to the fields using dotted notation.  However, that would make unsafe enum's syntax differ from the obvious expectation someone would have by looking at its definition, and dotted field notation doesn't make as much sense for safe, tagged enums.  A new construct doesn't come with those syntactic expectations (and anyone coming from C will expect a union to use struct-like syntax).
An unsafe enum with multiple fields would also adds to the complexity of supporting field access syntax: if one constructor of the unsafe enum looks like foo(u32, f32), how would you name the two sub-fields?  u.foo.0 and u.foo.1?  Most FFI interfaces name the sub-fields, which would require using a separate struct anyway; in that case, would you have to write u.foo.0.subfieldname?
Writing to fields seems similarly more complicated with enums.
As a minor additional nit, Rust warns by default for enum constructors that start with a lowercase character; many FFI interfaces would end up needing to disable those warnings.
I think the case of defining an inline structure would work better with an RFC for anonymous struct and union types; I'd be quite happy to write such an RFC as well.  Many FFI interfaces will want those anyway, for the common case of a struct containing an anonymous union.  However, I don't think that should form part of this RFC; I would suggest a followup after resolving this one.  In the meantime, it seems simple enough to define a struct (or tuple struct) and make that a field of the union.
All that said, I could live with unsafe enum or similar along with struct field access syntax; it doesn't seem as intuitive to me, but I'd take it over not having native support for unions at all.  (Only having pattern matching, by contrast, seems like a major wart for ergonomic union usage.)  As mentioned, I don't care that deeply about the syntax for declaring unions; I care a lot more about the syntax and semantics for using them.

It might be worth summarising some of the discussion from the internals thread - there is a lot of it and it's not easy to follow the threads of conversation.

As far as I know, I've captured all the major threads of discussion (including alternatives raised and the reasons for them) in the alternatives section.  If I've missed one, I can certainly add it.
The largest discussion was between unsafe enum and struct-like syntax (originally presented as #[repr(union)] struct, though I've now presented it here with a new keyword instead).

I would rather have tuple structs with a #[repr(union)] instead.

I am desperate to have untagged unions in some form to make my life easier, and I support this RFC being accepted and implemented as soon as possible.

Since forget is safe, how about allowing Drop types but guaranteeing that the destructors are not run? The the user can still use drop_in_place to manually drop the correct variant.

@mahkoh: I'd like to at least see a warning about it, absent some explicit indication of "meant to do that".  I don't deeply care whether it uses a warning or an error; @sfackler suggested making it an error for now, so I did.

@nagisa For FFI purposes, we need named fields.  A tuple version of untagged_union seems easy enough to support as well (using the same declaration syntax and field access syntax as a tuple struct), if you'd find that useful.  However, I'd like to avoid expanding this RFC for a first pass unless there's a strong demand for "tuple unions"; perhaps a followup RFC after this one?  The bigger this RFC gets and the more it adds, the more it takes to get accepted.

@joshtriplett that was more of a “instead of” comment. IMHO you should not expose these untagged unions to safe rust code anyway, so either way is fine, I guess, but I’m not sold by the named fields argument.
Also, for additional point (assumingly, a negative one), untagged unions is something @graydon said he was happy to see rust 1.0 to ship without.

All in all, my general opinion is that we should have some easier way to produce opaque untagged unions (essentially an opaque struct occupying as much space as necessary), perhaps implemented as a macro; but not the full blown way to do (and abuse) untagged unions.

@nagisa Macro solutions are what I already use for unions, and they're a pain to write and use and ensure that they are correct.

@nagisa: I absolutely agree that we shouldn't expose untagged unions to safe Rust code; only unsafe code can do anything interesting with them.
I've seen the macros @retep998 refers to; we desperately need a portable, safe solution.

That definitely suggests unsafe union. Null pointers and array "overruns", after all, are also not difficult to achieve in unsafe code, being similarly necessary for C FFI.

(The library I'm to interface with has essentially a tagged union { u31 direct; 0x80000000 | mpz_t* indirect }, where mpz_t is the GMP bigint struct. This RFC seems like the correct approach.)

@nrc

It might be worth summarising some of the discussion from the internals thread - there is a lot of it and it's not easy to follow the threads of conversation.

I skimmed the conversation but don't feel I can really summarize it. Still, here are some posts at least that I found interesting:

@joshtriplett on why identifying data simply by type may not be the best choice
@glaebhoerl on why piggybacking on struct or enum may not be the best choice

I've not yet read the final RFC to see where it ended up though!

Microsoft uses unions in two ways.

A union inside a struct which has a tag field. In this case the user would only need to access the union from the variant that it was initialized with. There are no doubts about this having well specified behavior.
A union that provides several representations of the same data. Such as having one variant be a UINT64  and the other variant a struct of four USHORTs. In this case, the user is expected to be able to write to one of the variants and read from the other variant. Even though the behavior of using a union like this isn't specified by some set of the C/C++ standards, it is specified by MSVC. Thus Rust would need to support it too.

Even though the behavior of using a union like this isn't specified by some set of the C/C++ standards

This is covered by footnote 95.

If the member used to read the contents of a union object is not the same as the member last used to
store a value in the object, the appropriate part of the object representation of the value is reinterpreted
as an object representation in the new type as described in 6.2.6 (a process sometimes called ‘‘type
punning’’). This might be a trap representation.

@mahkoh Thanks for looking up the exact reference in the C standard!  That's exactly the behavior I'd expect in Rust as well.

As I was mentioned here, some opinions (caveat: still not a core-team member, just opinions):

I'm fine with this as an unsafe FFI feature. I don't think anyone's proposing it as safe, so no problem.
I'd pretty strongly prefer doing it as an attribute like #[repr(union)] struct rather than a new keyword.

It is just a question of representation for a structure-with-fields. The representation is "the fields overlap". It's a weird fact of the representation, but memory changing underfoot is a fact of life in unsafe-land, for many reasons. Cf. multithreading, mmio.
Since they're inherently unsafe, the only function that ever sees them will be a wrapper that enforces safe access, and is very aware of their union-ness. Only the -sys code. They won't be drifting into safe library-user code. So I disagree that these will cause any user confusion
Yet more C-interop FFI tricks may yet wind up in repr(...); it's the right place for this. Cf. RFC 314, bitfields
I think the bar for new keywords is / should be extremely high, post-1.0
The question "what is the language versioning plan?" is interesting but unrelated since nobody asking for this feature would likely want to wait until 2.0, which might be years off. Meanwhile imo it's bad precedent to "allow new keywords, so long as they're suitably long/improbable/unwieldy".

@graydon I have no objection whatsoever to switching back to #[repr(union)] struct rather than a new keyword.

Didn't assume you did :smile: Just. Hrm. Sigh. Re-reading my words. I'm sorry, I need to figure out some way of participating in this community that doesn't come down like a ton of bricks / overstate my case all the time. I tried rewriting that 4 times and even still it's too pushy. Bleh. I realize my words carry more weight than they should. I haven't contributed a line of code in over 2 years now!

@graydon: I didn't assume you assumed I did. :)  I'm just trying to reiterate that I really don't care what color the declaration-syntax bikeshed is painted, just the syntax and semantics for usage.
Thus far, that declaration syntax seems like the bit that has produced the largest arguments.
I appreciated your comment, and I found it a compelling argument.  The only thing making me hesitate to change the RFC on the basis of that argument is that I've also received one vehement complaint (via IRC) against any syntax that uses struct.

I'd like to be clear that I was the one who argued vehemently against it on IRC. My argument was that unions are not structs, but another type entirely, much closer to an enum. An enum is, after all, just a tag and a union.
@Graydon there is a large difference between bitfields and unions. One is an (even more) packed struct. One is another way to transmute.
@joshtriplett LARGE_INTEGER is a good example of an API that requires you to be able to read it as any type. I am tending towards your original idea of it just being like a transmute_copy.

And just for variety I think unions are not at all like either structs nor enums. enums do happen to be represented as something like a combination of a union and a tag under the hood, but that is an implementation detail and has nothing to do with their behavior.
I believe the question we're trying to answer is "what should the language-level primitive be, and which other things should be layered on top in library code". And my sense continues to be that the most economic and "well-fitting" primitive would be something where you specify only the types participating in the union, and use as or transmute to pass both into and out of it. (This still leaves several different ways to formulate it; see the internals thread.) More specific things like assigning meanings/names to the types in the union, interacting with an external tag field, encapsulating safe access, etc. can be built on top of this using normal code, generated with a macro or FFI bindings generator, etc.

I haven't yet formed strong opinions, but some thoughts:

On the topic of #[repr(union)], the precedent of #[repr(packed)] seems worth noting -- we recently adopted (but have not yet implemented) RFC https://github.com/rust-lang/rfcs/pull/1240, which makes it unsafe to borrow fields of a packed structure. #[repr(union)] seems similar to me, if more extreme.
Another advantage to using #[repr] is that it makes this feel like a minor knob (which, to a large extent, it is) versus a keyword. Exhaustive lists of keywords and "kinds of user-defined types" and so forth often wind up listed in the front matter of books, which would put struct, enum, and untagged_union on equal footing. Doesn't seem worth it.
In general, I do think (also) that there is clearly a need for a feature like this, and it's time we took some RFC on the topic.
I prefer using structs, and hence field projection, as the basis for selecting a variant over previous proposals that used match, I think, because it seems like it more explicit about "selecting" a variant. Put another way, when I see a match I generally expect the compiler to be choosing an arm for me, and that is not at all what's happening here.
That said I was fond of @glaebhoerl's suggestion to somehow use union as Type, which seems a bit more obvious than union.field, but I also found @joshtriplett's rejoinder that field names (versus types) give more semantic information persuasive.

On the topic of #[repr(union)], the precedent of #[repr(packed)] seems worth noting -- we recently adopted (but have not yet implemented) RFC #1240, which makes it unsafe to borrow fields of a packed structure. #[repr(union)] seems similar to me, if more extreme.

For the record I thought (and still think) that's the wrong direction, but didn't think it was worth pushing back on too hard because it's such a small thing. But if small things get used as precedent to justify larger things that could change the calculus. (Though unions are still pretty small fry in the larger scheme of things.)
The main of appeal of using #[repr] for this sort of thing is that it minimizes the syntactic surface area (notably distinct from the full language-level surface area), but I think that valuing this too highly is misguided. Over the long term, shoehorning various pieces of indirectly related functionality into whichever existing constructs they can be shoehorned into with the least syntax will only lead to the language having a hacked-together feel when taken as a whole. On some level it may feel appropriate for FFI-related things to have this kind of feel given that they are interfacing with C, but I'm not sure if that's a beneficial mindset to have.
(None of that should be construed as an argument for the untagged_union keyword in particular, which I'm not particularly enthused by either. Its only real virtue is that at least it doesn't try to abuse repr.)

@glaebhoerl

The main of appeal of using #[repr] for this sort of thing is that it minimizes the syntactic surface area (notably distinct from the full language-level surface area), but I think that valuing this too highly is misguided.

I too am wary of overusing this sort of logic. However, this feels like a potentially appropriate place to me, in that it is ultimately a kind of corner of the language, one primarily concerned with legacy FFI interaction. That said, I think that while #[repr(packed)] is a kind of precedent, it is also different -- that is, removing implicit padding feels rather different from laying out the fields so that they overlap. (In particular, it retains the invariant that mutating a.b does not affect a.c, as you noted.)

@ubsan I do think it seems worth keeping the section in the RFC stating that you can't use unions to break the LLVM pointer aliasing rules (albeit I also think many people will not have any intuitive understanding of those rules).  However, if you don't have any further objection to using unions as transmute, I think I'll re-add a section that talks about writing one field and reading another.

As far as I can tell, it seems like we have at least rough consensus about every point except for the declaration syntax, and specifically the syntax to introduce a union declaration.  The two main contenders for syntax seem like #[repr(union)] struct or a dedicated keyword, though I'd certainly welcome a more appealing suggestion if someone has one.
untagged_union looks somewhat cleaner, but has the disadvantage of introducing a new keyword.  #[repr(union)] struct avoids a new keyword, and lets people immediately assume struct syntax; however, @ubsan objected to the use of struct.
@ubsan, how strongly do you feel about this issue?  And can you elaborate further on your objection?  I haven't seen anyone else object on the same basis you have, and I'd like to understand what particular problem you think will arise from this syntax.

Neither the behaviors of transmute, nor transmute_copy, nor any of the aliasing rules are properly specified in rust. Naturally you are not allowed to violate the rules anywhere, it's just unclear what the rules are. This doesn't just affect unions but all code.

Many unions in winapi have anonymous structs inside them and even anonymous structs inside anonymous unions. Currently I have to define structs with a name, like for a struct called CM_NOTIFY_FILTER with a variant called DeviceInterface I'd have the type for that variant named CM_NOTIFY_FILTER_DeviceInterface. Any proposal to make variant access of a union based on type rather than the field name would mean having to type CM_NOTIFY_FILTER_DeviceInterface instead of DeviceInterface to access that variant. I'd have to make a macro solution that provides inherent methods DeviceInterface() and DeviceInterface_mut() and that is exactly what I'm trying to move away from with this RFC! All I'd gain would be not having to manually calculate the size/alignment of the type, which on its own isn't really worth the hassle of new syntax. Having proper variant access instead of going through inherent methods created by ugly macro invocations is one of the key things that I need from untagged unions in Rust.

And let me repeat again that there are some problematic cases when there are union variants that are not repr(C). In particular, non-repr(C) objects don't officially have any kind of representation. So statements such as "the bytes are not changed", "the bytes are interpreted as", are not well-defined if you have non-repr(C) variants.

@mahkoh For non repr(C) types, it should definitely be undefined behavior to access a variant other than what the union was initialized with. I don't think anybody is trying to make that behavior specified or valid in any way.

@retep998 The current wording of the RFC does.

Then clearly @joshtriplett ought to fix that.

I'm very much against the untagged_union keyword. If necessary, union should be conditionally treated as a keyword based on a crate attribute.

@mahkoh How would that interact with the presence of APIs in libstd that have methods called union?

@retep998 By adding aliases with a different name.

Another possibility is to never parse union as a keyword after a ::.

Another possibility is to never parse union as a keyword after a ::.

Whether something is a keyword or ident is decided during lexing, not in the parser.

@nagisa Such implementation details are not part of the language specification.

I dosed with @mahkoh on reading a union variant should be unsafe. If I have a.rs:
'''rust
#[repr(union)]
pub struct Union {
x: u8,
y: u16,
}
pub fn foo(u: Union) -> u16 {
u.y
}

There isn't any way for the compiler to statically guarantee that won't access undefined data when it receives this Union from another crate or from an FFI call.

@erickt The RFC already states that field access on unions requires unsafe.

Such implementation details are not part of the language specification.

Implementation details not being part of language specification totally doesn’t mean we should ignore compilers 101 (i.e. traditional compiler frontend structure) and start adding special cases left and right when designing a given feature.

https://github.com/rust-lang/rfcs/pull/1083#issuecomment-95722858
Which one is it?

Which one is it?

I think we haven’t yet decided on an exact scheme, yet. Maybe it is a good time @nrc? Since this is not a first proposal in a pretty short timespan for a feature which is seriously considering a keyword of any sort?

the invariant that mutating a.b does not affect a.c

In unsafe code, this invariant (of structs) does not hold. Indeed it's explicitly broken in cases like writing threading primitives or mmio. Since we're talking about unions as an unsafe-only construct, I don't see it weakening the invariants one can assume about structs in unsafe-land.

For background, I am in the middle of attempting to define the language.
The specific complaints I have are few, but they would be the same for if you tried to make enum and struct the same kind of type.
struct S
    sizeof(S) >= Σ{sizeof(fields)}

enum E
    sizeof(E) = max({sizeof(fields)}, sizeof(discriminant)

union U
    sizeof U = max({sizeof(fields)})

As you can see here, the enum and union have far more in common than the struct. Second, how is access to them?
struct S { x: String, y: Vec<i8> }
enum E { x(String), y(Vec<i8>) }
union U { x: String, y: Vec<i8> }

{
    let mut s: S;
    s.x = String::new();
    s.y = Vec::new();
    // This next line could easily return a string or a vector
    s.x // String
    s.y // Vector
}

{
    let e: E;
    e = e::x(String::new());
    e = e::y(Vec::new());
    match e { // this can only return a vector
        y => y,
        _ => unreachable!(),
     }
}

{
    let mut u: U;
    u.x = String::new();
    u.y = Vec::new();
    // This can only return a vector
    // u.x - You can't do this
    u.y
}

As you can see, the semantics of unions are far closer to enums as well. They're just not structs. If we added this bodge to the language, it would be confusing and unfortunate, and I don't want to let that happen.

I don't hold any sort of strong opinion on the matter of repr(Rust) vs untagged_union vs unsafe enum. As long as I can easily define unions, access their variants by name, and have some sort of specified behavior when using them for type punning with repr(C) types, then I'm happy.

semantics of unions are far closer to enums as well

The semantics of unsafe unions (which are the only ones being discussed) do not guarantee any such things. Nor would structs in unsafe code. If you're assuming you can read back bytes you just wrote into a struct field, in unsafe code, you're assuming too much.

@graydon That is... kind of a ridiculous assertion, honestly. I'm not super sure how to respond to it. Except in the presence of (safe) threading, MMIO, or undefined behavior, writing to a struct field means that you can read from the struct field and get the same value back.

All concerns which are perfectly reasonable to assume in unsafe code.

I expanded my comment into an alternative RFC: https://github.com/rust-lang/rfcs/pull/1450

I guess I'm just curious what you mean by "attempting to define the language", if you're making such strong assumptions / leaving out such real-world considerations. It's some kind of formal model? I mean the sizeof() equations above also aren't right; rustc can choose quite a few representations for an enum or a struct. I think it'd even be legit for rustc to represent an enum as a struct, and just activate subsets of fields at any given time.

(Perhaps more to the point: why does the declarator syntax have such a direct relationship to your formal model? How can you build a formal model of unsafe code without considering the semantics a fair bit deeper than just the declarator syntax? In which case, why does the declarator syntax matter to you?)

@graydon Yes, it's a specification of the language. Declarator syntax matters because I want Rust to be the best language it can be, and this is an ugly bodge, that overloads struct to mean more than it should.
Threading also exists in safe code.
let a: AtomicUsize;
a.store(5, Relaxed);
a.load(Relaxed) // Could be literally anything
// NOT UNSAFE

This is in safe code, and you must opt-in, even in unsafe code. Same with MMIO.
Undefined behavior is invalid, and is not a part of this discussion. We don't have to deal with it because valid rust code does not use it.
If we had originally gone with data as the only declarator syntax, this would be a different discussion, but we did not. We chose to differentiate between struct and enum. And so, we must continue, because unions are not structs.

Unions are also not enums. :P Consider enums in C. They are merely an enumeration of disjoint nullary cases. We chose to piggyback on this intuition for algebraic sum types, calling them enums as well: they are an enumeration of disjoint cases, now potentially with associated data as well. If we take out the "disjoint" part and leave only the associated data (an untagged union), we're taking away exactly that part of it which justified calling them enums in the first place!

Therefore as a compromise, I modestly propose that we call them enum structs.

#[repr(union)]
enum struct Foo {...}
On Thursday, January 7, 2016, Gábor Lehel notifications@github.com wrote:

Therefore as a compromise, I modestly propose that we call them enum
structs.
—
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-169817287.

@ubsan I'm afraid this is getting rather heated. I never said anything about undefined behavior, and I don't think method calls add much to the discussion since they can do whatever they like to arguments. I'm just discussing whether a formal model of the unsafe fragment of rust's semantics depends on the declarator syntax of one of its constituents. I think it doesn't, so I think other concerns -- including intelligibility and aesthetics, but also compatibility, parsimony and precedent -- constitute the weight of the decision.
I'd also ask that you maybe tone it down, not just keep referring to things you dislike as "ugly bodges" as part of an argument. It's fair to have an aesthetic preference, or to find something confusing-to-read -- we all have our preferences and biases -- but that preference doesn't really get more convincing the stronger your choice of insult; it just adds heat to the fire and makes it harder to maintain calm when replying.

@graydon I'm sorry, I did get pretty annoyed. However, there is no good reason to make unions into a type of struct, and a lot of reasons not to.
@glabhoerl I'm not arguing that unions should be enums. I'm saying that if we were saying that unions absolutely could not introduce a new keyword, enums are the far more likely candidate.

I feel like @nrc's RFC makes this all much nicer, and much easier to work with. enums are the correct type to use here, but up until this point I haven't seen a way to use them.

@ubsan your size computation does not look completely right to me. I believe it should be
struct S
    sizeof(S) >= Σ{sizeof(fields)}

enum E
    sizeof(E) = max({sizeof(fields)}) + sizeof(discriminant)

union U
    sizeof U = max({sizeof(fields)})

(the difference is that in an enum you are not considering the discriminant in the max, you are adding it).
I think a possible interpretation is that an enum is a struct composed by a discriminant and an (unsafe) union. The discriminant is (automatically) used to unsure that the union is used in a safe way.

I feel like @nrc's RFC makes this much easier to work with

From the perspective of someone trying to specify the language or from the perspective of someone who is actually using it?

I like the concept in @nrc's RFC of making union variants their own types.  And as previously mentioned, I don't deeply care what the definition syntax looks like; if the definition syntax ends up using enum, I don't care that much.  However, using typecasting with as as the access syntax makes field access painful, especially by comparison to C unions.  Field writes, especially, become much more painful with this syntax.
Consider the very common case of a struct containing a tag and a union of other structs.  Consider in particular a write to a field inside one of those inner structs.  Consider the code volume and clarity with #1444 and with #1450.
Also, your comment in the "alternatives" section about "more unsafe code than you would like" doesn't seem clear to me; the amount of unsafe code required seems exactly the same between our two proposals.

Right now in winapi I define the union inline in the struct, with macros after the struct to define the variant methods. Any union RFC would mean I'd have to define the union outside of the struct, which is more verbose. For a union RFC to be usable for me it should at the very least remove the pile of macros I have to define the variant methods without causing the syntax to access the variants become significantly worse.
Direct field access would be ideal as the syntax would improve to C levels (imagine that, putting C as the ideal syntax), and would clean up that macro hell. @nrc 's proposal, if modified to auto import the variant when using as so I could always do foo as Variant without having to fully specify UnionName::Variant and without having to use UnionName::Variant, would provide syntax for variant access that isn't really worse than what I currently have, and would also effectively clean up the macro hell.
I cannot support any proposal which makes the syntax of variant access significantly worse, or doesn't improve it but neither does it make my life easier. Remember, the entire point of this union RFC business is to make working with unions easier and better, and if there's no appreciable improvement over my current macro solution then the RFC is effectively useless to me.

@joshtriplett

However, using typecasting with as as the access syntax makes field access quite painful, especially by comparison to C unions.

This is a good point. foo.bar.data compared to (foo as Bar).data is a big difference.

This is a good point. foo.bar.data compared to (foo as Bar).data is a big difference.

It's even worse for variants that just hold a single field: foo.bar compared to (foo as Bar).0.

@graydon
I agree with @ubsan that structs are a suboptimal choice for unions, but I feel it may be worthwhile to explain my reasoning.
Specifically, my concern is about the programmer's intuition, and how much out-of-band knowledge we expect them to have, particularly knowledge not relevant to their current task.
With structs, your examples of how memory can change out from under the programmer are special cases of interacting with unusual kinds of memory (mmio) or raw details of the execution model (unsafe threading). The user's default assumptions with structs are generally that they're a product type; that is, the freedom it has wrt values is the product of the freedoms of the members.
With enums, the user is already in the mindset of "the value is only one of the variants" - their intuition already pushes them away from trying to mix access to two variants, without requiring they consider something beyond their actual use case (FFI, or possibly doing unsafe type handling).
So while it's true that, in the unsafe setting, struct fields can change out from under the programmer, I don't think that's the most useful framing of the behavior. To me, it seems counter to the Rust desire that users "fall into the correctness pit" - a goal that isn't invalidated by being inside an unsafe { } block.

@eternaleye said it much better than I ever could. Thank you.

I have decided I was wrong and that unions are better represented as structs than enums. The reason is that accessing unions with field access rather than casting (itself an innovation) is more ergonomic in all circumstances I can think of. The sole advantage of the enum approach seems to boil down to declaring a whole data structure inline, which seems not so important.
My intuition now is that the distinguishing feature of a data type should not be its representation but its access pattern - accessed via matching means it's an enum, accessing via field access means it's a struct. We can't match on a union, and field access is better than casting. Put another way, we layout structs in many ways already, we are just adding another way (albeit a more dramatic change). Others have said all of this (and better) in this thread.

@nrc

I have decided I was wrong and that unions are better represented as structs than enums. The reason is that accessing unions with field access rather than casting (itself an innovation) is more ergonomic in all circumstances I can think of.

I think I agree as well. At first I was surprised by the idea of using a struct as the basis rather than an enum, but I've gradually become swayed by a few factors:

@graydon's point about how union is just a struct with the fields laid out at the same spot
the fact that in C a struct and a union declaration feel more-or-less analogous, albeit with distinct keywords
ergonomic concerns
the match vs project point -- projecting out a field feels like an active choice of which variant, whereas match does not (as you mentioned)

I still generally feel that adding a third category of item (e.g., a union keyword) is overkill. Untagged unions just don't feel like they merit such first class treatment to me.

I still generally feel that adding a third category of item (e.g., a union keyword) is overkill

Strongly agree. I think #[repr(union)] struct Foo { ... }` is the right syntax.
If we think of struct as being about access, then the difference between a standard stuct and a union is just a matter of representation (of course this affects other things (unsafe field access), but that is true of #[repr(packed)] too).

@nrc: well, there are two sides to a datatype: construction and observation (access). A union is a datatype that looks like an enum when you construct it, and that looks like a struct when you access it.
I like the as approach to union - on a casual browse, .foo looks like you're doing absolutely nothing special, and in particular absolutely nothing unsafe or dangerous.

@lilred Well, *foo doesn't look special, either, but it's unsafe for raw pointers (but not references or Deref implementers). There's also no such thing as an as cast which requires unsafe yet, so making a field access unsafe doesn't seem particularly worse to me.

Quiz: in the following code, which is the unsafe access?
fn frobulate () {
    unsafe {
        foo.bar.baz.qux();
    }
}

    fn frobulate () {
        unsafe {
            foo.bar().baz().qux();
        }
    }

I'm not sure what you're saying.

Which is the unsafe function call.

Whenever you have multiple operations in an unsafe block, regardless of whether they are field access or calling a function or even borrowing something (due to repr(packed)), it can be difficult to figure out which thing actually needs the unsafe.
What @mahkoh is saying is that just like it can be difficult to figure out which field access is unsafe, so it can be difficult to figure out which function call is unsafe. If you want the code to be clear, only put the unsafe operations in the unsafe block.

I'd prefer a design that isn't C-like unsafe transmutation, but more Rust-like: an enum, but with user-defined implementation of tagging.
In C unions are used for two purposes: tagged union and transmuting/casting. Rust can do transmuting already, so I think the problem can be narrowed down to tagged union where the tagging method is user-defined and arbitrarily complex.
Could this functionality be satisfied with an enum that has user-defined binary representation?
enum {
    Integer(u32),
    Float(f32),

    // I'm not sure about details of the syntax, but the intention is to have custom functions that handle tagging details, so that the enum can be made as usable as a Rust enum.
    fn set<Integer>(&mut self, value: u32) {…}
    fn get<Integer>(&self) -> Option<u32> {…} // or…
    unsafe fn get<Integer>(&self, out_of_band_information: &Foo) -> Option<u32>  {…}
    fn set<Float>(&mut self, value: f32)  {…}
    fn get<Float>(&self) -> Option<f32>  {…}
}

There's probably a better way to specify the interface of getters/setters (traits? OIBIT? lang item? dunno).
But in general it'd be cool unions were enums, merely with the default built-in implementaiton of getters/setters for tagging, but the implementation could be customized.

I really liked the Ada-like layout proposal I heard on whichever discussion that has been happening around. My idea for enums using something like that would be:
struct MyAwesomeTransmuter {
    #[layout(0..64)]
    full: u64,
    #[layout(0..)] // either end can be left off and also allows for incorporating bitfields.
    lowerhalf: u32,
    #[layout(..64)]
    upperhalf: u32
}

Here is the baseline:
#[repr(C)]
struct MyUnion {
    _data: OpaqueUnion<T1, T2, T3>,
}

impl MyUnion {
    fn t1(&self) -> &T1 { mem::transmute(self) }
    fn t1_mut(&mut self) -> &mut T1 { mem::transmute(self) }
    fn t2(&self) -> &T2 { mem::transmute(self) }
    fn t2_mut(&mut self) -> &mut T2 { mem::transmute(self) }
    fn t3(&self) -> &T3 { mem::transmute(self) }
    fn t3_mut(&mut self) -> &mut T3 { mem::transmute(self) }
}

fn f(x: &mut MyUnion) {
    x.t2_mut().field = 1;
}

If your proposal is more complicated than this, it's worthless. Why? Because the code I just posted can (almost) already be written today and nobody will use a feature that does the same but is more complicated. And once the type OpaqueUnion can be written, your feature might as well be deprecated.

@nagisa That's makes repr(C) useless because you have to manually compute the layout. Which is already possible today.

@mahkoh couldn’t a
union {
    T blah,
    TT blahblah,
    TTT blahblahblah
}

be equivalent to
#[repr(C)]
struct {
     #[layout(0..)]
     blah: T,
     #[layout(0..)]
     blahblah: TT,
     #[layout(0..)]
     blahblahblah: TTT
}

That proposal is much more complicated than what is needed for unions and it's not clear how/if anything but the 0.. case should/could even work. Write the details down so that we can evaluate it properly.

@nagisa I think that such control for the layout of Rust structures might be interesting, but it is not going to be a convenient way to express the C union types used in FFI (I believe that the main thing needed to make them useful is to rely on the compiler to use the appropriate ABI/layout automatically). Maybe that should be proposed in a separate RFC.

So does this RFC subsume https://github.com/rust-lang/rfcs/pull/197 (ManuallyDrop)? @huonw, thoughts? I think it was @glaebhoerl who first suggested that a union might fulfill the same use case, and this RFC seems to provide a particularly ergonomic way of doing it.

As far as I can tell, a univariant union would be effectively a ManuallyDrop and then it would simply be a matter of adding a convenient ManuallyDrop wrapper to std which could just as easily be done in a third party crate.

@retep998

As far as I can tell, a univariant union would be effectively a ManuallyDrop

Unfortunately, I think the RFC as written does not allow for that. In particular, the RFC reads:

The compiler should produce an error if a union field has a type that implements the Drop trait.

I think I would prefer to remove this line, so that unions can subsume #197. I'm curious to hear what others think on this topic (particularly @huonw)?

A few more thoughts about this line:

The compiler should produce an error if a union field has a type that implements the Drop trait.

This line is actually not specific enough to be readily enforceable. What if there is a generic union? We can't decide if this type requires Drop or not:
#[repr(union)] // I still prefer this syntax :P
struct Foo<T> {
    x: T
}

So, to enforce this line, we'd have to make it a well-formedness condition of a particular application of Foo to some type. e.g., Foo<Box<i32>> would be an illegal type, but Foo<i32> is ok. However, that in turn interferes with potential extensions to HKT.
All this is to say, ad-hoc rules about what kind of types are permitted (typically) don't really work. To make it all fit in, you would really need to require a trait bound. The opposite of Drop is Copy, so you could add that here, but obviously nobody wants that.
Since mem::forget is safe, it is still ok for creating a union to be considered safe (even though it can easily trigger a leak).

@joshtriplett
I looked over the RFC again just now and had one minor question.

Unsafe code may pattern match on union fields, using the same syntax as a struct, without the requirement to mention every field of the union in a match or use ..:

Does this mean that a pattern like MyUnion { x: .., y .. } is legal? (This would presumably interpret the bytes first as x and then again as y?) I would have expected that a union pattern can have at most one field, kind of like a union constructor (except that requires exactly one field).

@joshtriplett Ah, I see you had some unresolved questions on that topic. On the topic of this question:

Can the borrow checker support the rule that "simultaneous borrows of multiple fields of a struct contained within a union do not conflict"? If not, omitting that rule would only marginally increase the verbosity of such code, by requiring an explicit borrow of the entire struct first.

I guess you are referring to the second example you gave in the borrowing union fields section? That should not be a problem, I wouldn't think, though it'll require some special logic to achieve (scenarios like that don't arise currently).

The compiler should produce an error if a union field has a type that implements the Drop trait.

@nikomatsakis I believe the intention for that line was to simply be a lint that was warn or error by default.

@retep998 Right; even with it as an error, it seems easy enough to have a language extension that disables the error, if you have a module that intentionally wants to put a type with Drop into a union.

@nikomatsakis Thanks for the answer about the borrow checker.  I'll update the RFC accordingly.
Regarding pattern matches of multiple union fields simultaneously: my original RFC specifically allowed that, but subsequent discussions about aliasing rules and similar led me to drop it.

So, at this point, I'd like to summarize what look like the three remaining key points of disagreement, and make them as orthogonal as possible.

People disagree whether to model union field access syntax after struct or enum.  Here, I consider practical usability and ergonomics the most compelling argument: unions only make sense to add if they make code significantly better than that achievable via macros or generics.  So, for the purposes of this RFC, I intend to maintain struct syntax for field accesses, with other syntaxes left to the "alternatives" section.

People disagree whether to model union declaration syntax after struct or enum.  Here, the arguments about usability and ergonomics don't apply as strongly, as declaration occurs less frequently.  Personally, I believe that given the use of struct syntax for (1), using struct syntax for the declaration makes sense for consistency, and I don't plan to change the RFC to use enum declaration syntax.  However, if Rust ends up with enum declaration syntax and struct field access syntax, I would still consider this effort a success, albeit quirky.

People disagree whether to use a new keyword (whether untagged_union or something else) or whether to use some construct that doesn't require a new keyword (whether #[repr(union)] struct or something else).  I personally don't care about this at all, and I'd like to avoid bikeshedding here.  Given (1) and (2), I don't consider the use of struct in #[repr(union)] struct an issue.  So I think this really comes down to "do we add a new keyword or not?".  I would happily present those two possibilities to the language committee as equally viable choices, but an RFC needs to make a specific proposal.  So, having read and considered all the arguments on this point, and in the absence of any clear information about Rust policy for new keywords, I plan to change the RFC and this pull request back to proposing #[repr(union)] struct, and move the proposal for a dedicated keyword to the top item in the "alternatives" section.

@joshtriplett that all sounds good to me. On point 3, a reason to support #[repr(union)] struct rather than a new keyword is that we want unions to be somewhat second-class. I don't think a programmer should ever consider using a union unless necessary, so there should not be a choice between enum and union. Unions should only be used when they are needed.
I think the semantics around destructors are important to get right (cc @nikomatsakis). My preference is that programmers should opt-in to ignoring Drop implementations (e.g., by using ManuallyDrop). However, the question of generics is a bit tricky. I imagine we could check wherever a union type is instantiated, but that feels a bit yucky. Failing a solution to the generics question, I guess ignoring destructors would be ok.

@joshtriplett The issue with #[repr(union)] struct that I have, (besides the fact that unions aren't structs), is that the only way that unions follow structs is in access. They follow enums in creation.
untagged_union LARGE_INTEGER {
    u {
        low_part: u32,
        high_part: i32,
    },
    quad_part: i64
}

Creating one of these would look like this:
{
    let li = LARGE_INTEGER { quad_part: 5 }; // this looks like a struct, but it's initialized after this
    // You don't have to initialize all fields, and in fact can't. In direct contrast with all other structs
    li.u.low_part // this is not an error!
}

{
    let mut li = std::mem::uninitialized();
    li.u.low_part = 5;
    li.u.high_part = 0;
    li.quad_part // not an undef value, despite never writing to it
}

They follow enums in creation.

@ubsan While this is true, I think what @joshtriplett said about declaration also applies to construction:

People disagree whether to model union declaration syntax after struct or enum. Here, the arguments about usability and ergonomics don't apply as strongly, as declaration occurs less frequently.

Access is just so much more common than declaration and creation that the ergonomics for access need to be prioritized.

I would expect the compiler to never generate a Drop impl for a union, and instead it must be manually implemented.

@tsion I would rather not have unions in the language at all, and have them be a library, than overload the struct keyword in this way. It's misuse of a data type. unions aren't structs.

@nrc I'll keep the statement that the compiler should attempt to produce an error on code that puts a type with Drop into a union, but phrase it as a best effort rather than an absolute; I intended it as a diagnostic to help avoid obvious errors, not a fundamental soundness requirement.

@joshtriplett I would prefer a diagnostic rather than an error. Warn-by-default, rather than deny-by-default.

If it's just going to be a best effort, then just make it be a lint that is either warn or error by default, and explicitly say it will just be a lint in the RFC.

@nrc The next version will explicitly use the term "lint", and leave open the possibility of disabling that lint.
@ubsan (In response to your comments about struct.) While I disagree with you, I do not intend to just dismiss your argument, and I will endeavor to cover it fairly within the "alternatives" section of the RFC.  As you've pointed to @eternaleye's argument making the point about product types versus sum types, which makes sense to me as well from a type theory perspective, I'll use that to attempt to explain this concern; once I've pushed the new version, I would welcome any suggestions to strengthen that explanation.
On the specific point of the introduction of a keyword, I find the arguments against introducing a keyword ("don't break existing code") more compelling than the arguments in favor of one; furthermore, if a keyword policy arises in the future, we can always introduce a new keyword-based synonym at that time, and I'd highly recommend doing so.  Once past that question, in the absence of introducing a new keyword, I do not see a proposal on the table that satisfies your qualms about the term struct while simultaneously providing maximally usable syntax for declaration and usage.  Every other proposal I've seen sacrifices a significant amount of readability and usability, or introduces a significant likelihood for programmer confusion (such that if a programmer does the most obvious thing they'll get it wrong).
I have a hard time imagining a user looking at a declaration that says #[repr(union)] struct, ignoring the pragma, and thinking it should behave exactly like a struct.  (Furthermore, #[repr(packed)] struct provides precedent for a pragma giving a struct an unsafe representation.)
I would very much welcome any concrete proposal to replace #[repr(union)] struct or untagged_union with something that 1) doesn't introduce a keyword, 2) doesn't use struct, and 3) doesn't use a declaration syntax inconsistent with its usage syntax.
While the grammar at https://doc.rust-lang.org/grammar.html seems somewhat incomplete, based on what I see there, it doesn't seem possible to use a pragma alone to distinguish between specific grammar productions.  And introducing a "pseudo-keyword" or "unreserved keyword" seems problematic.  unsafe #[union] could work, since right now an "attribute" (pragma) can't appear immediately after unsafe at the top-level, but that would complicate the production for unsafe fn.  Looking at https://doc.rust-lang.org/grammar.html#keywords , the least objectionable thing I can find would look like as #[union] (or any other keyword like as that doesn't currently mean anything as a top-level item), but since the parsing can't currently depend on the contents of the attribute, that would lock a top-level use of that keyword (e.g. as) into introducing a type declaration with a specific syntax production; that seems quite problematic.
One idea that might potentially work: what if we introduced a new top-level "macro" implemented entirely by the compiler, as a syntax extension rather than using macro_rules!?  In particular:
union! U {
    i: i32,
    f: f32,
}

This should parse correctly using the current rules for macros (especially considering that it follows the same pattern as macro_rules!, namely an identifier followed by a brace-delimited series of tokens with proper delimiter pairing), and all of the remaining constructs associated with unions should already parse correctly.
As far as I can tell, new versions of Rust do introduce new non-keyword names in existing modules, even though rustc will complain if you define your own function with the same name.  So if someone defines a function named "memfrob", and also has use std::mem::*, they'll break if the standard library introduces std::mem::memfrob.  Given that, does this proposal seem more reasonable?  It addresses the concern about introducing a new keyword, and avoids using the word struct anywhere.

Posting this as a separate comment to avoid burying it at the end of my reply.
One idea that might potentially work: what if we introduced a new top-level "macro" implemented entirely by the compiler, as a syntax extension rather than using macro_rules!?  In particular:
union! U {
    i: i32,
    f: f32,
}

This should parse correctly using the current rules for macros (especially considering that it follows the same pattern as macro_rules!, namely an identifier followed by a brace-delimited series of tokens with proper delimiter pairing), and all of the remaining constructs associated with unions should already parse correctly.
As far as I can tell, new versions of Rust do introduce new non-keyword names in existing modules, even though rustc will complain if you define your own function with the same name.  So if someone defines a function named "memfrob", and also has use std::mem::*, they'll break if the standard library introduces std::mem::memfrob.  Given that, does this proposal seem more reasonable?  It addresses the concern about introducing a new keyword, and avoids using the word struct anywhere.

Earlier @mahkoh asked why we don't just parse union as a keyword based on context, and I'm wondering the same thing.
Even with union as a contextual keyword (still allowed as a user identifier), unsafe union Foo { ... } seems like a workable grammar. At module scope, after seeing unsafe the parser is only looking for fn or union as far as I know, and in block scope a "statement" beginning with unsafe could only be followed by { (making it an unsafe block statement) or fn (making it an item) currently, so adding a case for union there would be fine.
Come to think of it, even union Foo { ... } might work with union being contextual, since no statement can legally begin with something like union Foo {. There could be union.foo, union + bar, or any number of expressions, but union Foo { isn't current valid anywhere an item definition is allowed. Correct me if I'm wrong.
However, I don't know the policy for adding contextual keywords. Are they really so problematic or are they a real possibility? I can think of multiple pros and cons, but I don't know if this is the best place to go off on that tangent.
@joshtriplett At any rate, it's probably worth noting that the keyword alternative could be a non-breaking addition unless I'm missing something.

@tsion @joshtriplett I think both the macro_rules! solution and the contextual keyword solutions could work, and they're both far better and look nicer than the current solutions.

@tsion I have that idea of "unsafe union" mentioned in the alternatives; it seems perfectly reasonable if the parser can handle it, which I'll leave up to parser experts.
@ubsan Thank you!  Working on the rewrite now.

@joshtriplett I was thinking more about the borrow checker integration, and
I realized that there is a bit more uncertainty than I originally thought
as to the correct behavior. Imagine that you have a setup like this:
#[repr(union)]
struct Foo { a1: A, a2: A, b: B }

struct A { f: u32, g: u32 }
struct B { h: u32, i: u32 }

The RFC suggests that&mut foo.a1.f could co-exist with &mut foo.a1.g,
but what about &mut foo.a2.g? And presumably it would be an error for it
to co-exist with permit &mut foo.b.h or &mut foo.b.i?
In other words, I presume that what would matter is the type of the field
(A or B) and not the name of the field, in terms of what the
borrow-checker permits.
On Mon, Jan 18, 2016 at 6:57 PM, Scott Olson notifications@github.com
wrote:

Earlier @mahkoh https://github.com/mahkoh asked why we don't just parse
union as a keyword based on context, and I'm wondering the same thing.
Even with union as a contextual keyword (still allowed as a user
identifier), unsafe union Foo { ... } seems like a workable grammar. At
module scope, after seeing unsafe the parser is only looking for fn or
union as far as I know, and in block scope a "statement" beginning with
unsafe could only be followed by { (making it an unsafe block statement)
or fn (making it an item) currently, so adding a case for union there
would be fine.
Come to think of it, even union Foo { ... } might work with union being
contextual, since no statement can legally begin with something like union
Foo {. There could be union.foo, union + bar, or any number of
expressions, but union Foo { isn't current valid anywhere an item
definition is allowed. Correct me if I'm wrong.
However, I don't know the policy for adding contextual keywords. Are they
really so problematic or are they a real possibility? I can think of
multiple pros and cons, but I don't know if this is the best place to go
off on that tangent.
@joshtriplett https://github.com/joshtriplett At any rate, it's
probably worth noting that the keyword alternative could be a
non-breaking addition unless I'm missing something.
—
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-172687576.

@nikomatsakis Neither the type of the union field nor the type of the fields within it matter when checking borrows.  The borrow checker should not attempt to determine if union fields overlap or if they have the same type.  In the example you gave, after borrowing &mut foo.a1.f, you can borrow &mut foo.a1.g, but you cannot borrow any part of either foo.a2 or foo.b.
Borrowing both fields of a1 has the same semantic as first borrowing a1 and then sub-borrowing its individual fields.

Odd that github doesn't show the big commit to change syntax to union! U { ... }.  In any case, editing the initial request now.  (Still on a branch named "untagged_union", sadly, because github doesn't support changing the branch for a pull request.)

@ubsan Untagged unions (where there's truly no tag, not even out of bound) to me seem merely like a syntax sugar for transmute. Are there cases where transmute can't do the same thing?
trait LargeInteger {
    fn u(&self) -> &HiLo;
}

struct HiLo {
    high_part: i32,
    low_part: i32,
}

impl LargeInteger for i64 {
    fn u(&self) -> &HiLo {
        unsafe {std::mem::transmute(self)}
    }
}

fn main() {
    let x:i64 = 5;
    print!("{}", x.u().high_part);
}

@pornel Your code fails to take alignment into account, so HiLo might only require an alignment of 4 while i64 requires an alignment of 8. This doesn't matter for going from i64 to HiLo, but as soon as you go from HiLo to i64, if it doesn't happen to be 8-aligned, it can crash on some platforms.
union is handy, then, since it takes the maximum of the alignments of its fields. Similarly, it provides enough space for the largest field which can be important e.g. when mutating a union in-place to become a variant of a different size.

@pornel std::mem::transmute can implement this, but the code doing so requires platform-specific knowledge of structure layout, alignment, padding, etc.  Folks in this thread, such as @retep998, have written macros that implement unions, with access functions to read and write fields.  In addition to producing much more painful syntax for field access, the portability issues even just between 32-bit and 64-bit Windows make this bad enough to motivate a native solution, even without throwing in a dozen more target platforms/architectures.
The compiler already has all that information so it can lay out structures; reusing that information for unions makes sense.  And doing so allows much simpler field access syntax, including reads, writes, initialization, and pattern matching.

@joshtriplett

Neither the type of the union field nor the type of the fields within it matter when checking borrows. The borrow checker should not attempt to determine if union fields overlap or if they have the same type

Yes, OK, this seems best.

I remain pretty uncomfortable with adding union as a third category on equal footing with struct or enum. I just don't think they merit that status. I think they can be nicely understood either as oddball structs or oddball enums. In the case of structs, they can be understood as a case of structs with a rather particular layout -- all fields overlapping. In the case of enums, they can be understood as enums without a discriminant. I prefer structs because field projection seems like the superior way to select the variant.
There is also the matter of their inherently limited utility. Unions will always be confined to unsafe code -- FFI interop is obviously a prime use case, and I imagine they'll be used both to handle the cases described in the ManuallyDrop RFC #197 and for other "space-saving" sorts of things. I feel like we don't need to use "language real estate" as readily for this sort of thing -- it's ok to stretch an existing definition instead.
I am particularly uncomfortable with using union!, for a number of reasons: in general, we only use ! for user-defined macros or things that could be made into user-defined macros. Users could not define a macro that looks like union! foo { ... } today, though I'd like to change that. But even if they could, it'd be impossible to write a macro that behaved like union!, since it transforms ordinary field access and requires deep integration with the borrow checker. So it basically just seems misleading to use the ! -- the only real reason to do so is "backwards compatibility", and it's not even necessarily backwards compatible, since there may be existing macros named union! (obviously, those could shadow the new definition, but given the fact that macros have global scope, that seems suboptimal). Basically, I would like to stick to the rule that things which use the ! are pure front-end procedures that could in principle be written as a macro.
All that said, I can certainly understand the discomfort around using #[repr(union)]. In general, my rule of thumb for when #[repr] is appropriate is that it should not transform the type into something completely different -- operations which remain legal should still have the same semantics (except possibly faster or something). #[repr(union)] doesn't really follow this rule: you could almost justify it because basically all accesses become unsafe, and hence no existing code would continue compiling, but that feels like a violation in spirit. =)
(And yet, I still feel like #[repr(union)] just squeaks by in terms of being acceptable, but I'm hard pressed to say why, beyond the reasons I wrote above and what @graydon articulated: basically -- in unsafe code -- one could certainly encounter ordinary structs that behave the same way, if you have aliasing pointers or threads involved. The fact that Ada allows custom layout also seems to match this intuition.)
The other thing I have wondered is whether we will ever want to apply a #[repr] attribute to a union. For example, would it make sense to have #[repr(packed)] on a union? Or to adjust its alignment? It seems like you can always do that by adjusting the layout of the constituent fields, but I'm not sure.
I'm trying to think of a "third way" here. One compromise I might consider would be using union as a a contextual keyword like union struct, though I'm still not super keen on that. I admit that unsafe enum feels like a natural declaration syntax, but I also am persuaded that field projection is the superior basis and that doesn't seem to mix with using the enum keyword. Have to think on it.

@nikomatsakis:
What if unsafe enum generated intrinsic-impl accessor methods that are fn to_snake!(Variant)(&mut self) -> &mut Variant; ? With or without some #[derive(Union)] annotation?

An alternative that came up while spitballing with @pnkfelix (who, btw, disagrees with me about #[repr(union)]) would be that we could declare an unsafe enum but permit field projection using the variant names. e.g.,
unsafe enum Foo {
    Bar(i32),
    Baz(i32)
}

and then you would write foo.Bar.0. Not sure if this is a good idea, but I feel like I hadn't seen it (or hadn't seen it recently).

@nikomatsakis Not sure if that came up in this thread, but it's been discussed a bit on the discourse thread and on IRC. It's essentially the same as the (foo as Bar).0 proposal and thus also requires something along the lines of @nrc's variant types proposal because, for example, what is the type of foo.Bar?

@nikomatsakis We have a significant amount of demonstrated demand for union support, both for FFI and for implementations of space-optimized structures in Rust; I think it's worth pitching an RFC for native support.  Given that, I don't see the benefit in making the syntax intentionally more second-class; this doesn't reserve any more or less "language real estate" whether it uses union! or #[repr(union)] struct or some other syntax.
Regarding the use of union! in particular, note that "macro_rules!" cannot be written as a Rust macro either.  Regarding backward compatibility, any Rust code that defines its own union! macro will never conflict with the built-in one; it'll always use its own local definition.  I don't see any way that a built-in union! macro could break any existing Rust code.
I listed in the alternatives section the idea of using the compound keyword "unsafe union"; that would work just as well, if the parser can handle it.
@eternaleye Accessor methods would result in much uglier code than struct-style field access syntax.
@nikomatsakis That idea came up multiple times, in the context of a few different proposals, including @retep998's unsafe enum proposal and @nrc's proposal.  (Both of whom have endorsed this RFC instead.)

@joshtriplett

this doesn't reserve any more or less "language real estate" whether it uses union! or #[repr(union)] struct or some other syntax.

I am not sure. I think there is a real difference in "complexity feel" if things are implemented using special syntax, for example, or a library -- and I think attributes and modifiers fall in between. Perhaps I am projecting (no pun intended), but they feel to me more like details that don't need to be fully understood at first. For example, the #[repr] attribute is something you only need to know about when you start moving beyond the platonic ideals of product and sum types and into the real world of bytes in memory.

Regarding the use of union! in particular, note that "macro_rules!" cannot be written as a Rust macro either.

This might be a matter of using distinct terminology. When I say "macro", I really mean "bit of code that runs in the front end" -- it might be a "syntax extension" (i.e., plugin) or it might be a "macro rules" macro. And certainly our goal has always been that macro_rules could be written as a syntax extension, once we make syntax extensions powerful enough. And even if it winds up not quite being true, it is very close to being true -- the effect of macro-rules is quite local to the very beginning of compilation.
I guess another way to put it is this: if the objection to #[repr(union)] is that "a struct is a product type, and a union is not" (which, as I said, I agree with), then moving to a macro seems to just raise a similar objection (a union! is not a macro). In particular, union! cannot be "desugared" in some local way to some other constructs that will behave the same way.

That idea came up multiple times, in the context of a few different proposals, including @retep998's unsafe enum proposal and @nrc's proposal. (Both of whom have endorsed this RFC instead.)

I believe that @nrc's proposal used the notation (foo as Bar).0, instead of foo.Bar.0. But I wouldn't be surprised if this suggestion has come up -- we've been talking about unions for a long time! :)

@nikomatsakis Well, then perhaps this is an argument for unsafe union, then. IMHO, we shouldn't special case structs all over the place in order to support unions (because that is exactly what we would have to do. You have to explicitly initialize all fields of a struct (except for #[repr(union)]). Writing to one field won't change another (except for #[repr(union)]). etc., etc.)

I agree with @ubsan; syntax aside, the compiler will need to treat this as a new construct internally, rather than a struct or enum, because it behaves differently than both and provides different semantics than both.  Whether we use union!, unsafe union, or something else, the underlying implementation in the compiler should remain the same.

@nikomatsakis Today, Rust has extensive FFI features, but you don't need to pay attention to them unless you build or work with code using them.  The same would apply to unions, whether implemented via union!, unsafe union, or an attribute.  Semantically, they represent exactly the same concept to understand, just with a different set of tokens introducing a declaration.
Apart from that, I agree that the compiler does not today provide enough functionality to implement union! in the language itself; if it did, this RFC wouldn't be needed.  I'd love to see enough functionality in syntax extensions to change that.  But apart from that, this just seems like a question of naming.  And union! has the advantage of backward compatibility.  unsafe union would as well if the parser can handle it.  Either seems better than an attribute on struct or enum.

There is also the matter of their inherently limited utility. Unions will always be confined to unsafe code

They can be, but they don't really need to be - there have been a few proposals in the past for some sort of "true POD" trait to allow transmutation in safe code, and unions would just be a special case of that.  (There would probably need to be some kind of attribute to force zero initialization of padding bytes...)

@comex That is true. That is unlikely to be something we will do, however, because the only case that will be useful is with something like LARGE_INTEGER, and you're already going to be using unsafe with FFI like that.

@ubsan Shrug.  Unions can also be useful for space optimizations, although if you really are just storing integers, you can probably get the same code generated in most cases by doing manual bit shifting etc..  However, the sort of trait I mentioned would be fairly useful for dealing with binary formats etc., even if it could only be used with some sort of transmute function rather than unions: while most users should probably look to a library that can do things like endian conversion automatically, transmuting can be a significant performance win, and heaven knows file format parsing code has greater need for safety than most.  If that machinery is ever created, extending to unions would be pretty trivial.  Anyway, something for the future.

@comex re: "Doing things like endian conversion automatically" - casting bytes to integers and then swabbing is in pretty much every case suboptimal, and often actively harmful. I'd suggest http://commandcenter.blogspot.com/2012/04/byte-order-fallacy.html - it's a very well-done writeup.

@eternaleye I didn't go into enough detail previously but that's pretty tangential.  In fact, Rob Pike's post doesn't even really address the real issue in C causing unnecessary use of things like byte swaps: C makes it very easy to decode binary data by just casting it to a struct, but doing it in a more abstract way requires implementing or importing a custom little binary data library, because the standard library doesn't have any alternative.  Meanwhile, traditional languages don't have any remote equivalent, so you end up with library functions like Python's struct.unpack, or "reader" classes with methods like readUint32(), etc. (or higher level abstractions like Python's Construct, but those are rarer).  But even in C, using the latter sort of abstraction for reading and writing binary data has many advantages, and while casting is faster, the difference isn't meaningful to most applications.  Those advantages include making it much easier to get bounds checking right, allowing for more sophisticated binary formats and/or less manual code to handle non-fixed-length data structures, and yes, banishing endian issues, but only as one of multiple portability concerns such as struct layout and unaligned accesses.  Focusing on endian alone is missing the point.
However, if, hopefully for actual measurable performance reasons, you do want to directly access the fields of the original data rather than copying, you either (1) cast to a native struct type or (2) use some macros or verbose functions to recreate a portion of the type system.  The latter is perfectly viable, and indeed could be done well in Rust with compiler plugins, among other things making it a bit easier to generate code to automatically verify the format, dump it, etc.  But using native types has advantages too.  In that case, explicit endian swapping can still be avoided by declaring your struct fields as some custom type like BEU32, with get() and set(); whether that type is implemented in Rob Pike's suggested way or with swapping is not very important.
(Albeit of more than stylistic importance: while one would think all modern compilers would detect the byte-at-a-time integer load pattern and replace with a native byte swap or big-endian load instruction where appropriate, LLVM currently does not.)
I'm sorry, this post is a total digression.

The whole discussion goes in a very wrong direction IMO.
There's no need in native Rust unions, there's no need in well-integrated convenient access syntax, unions are a very niche very FFI-specific feature.
Give FFI-people 1) a way to create a properly sized and aligned piece of storage 2) a way to auto-generate accessor methods to it with something like #[derive] and then 3) leave them on their own.

@petrochenkov So with point 2 you propose having a built in macro which creates foo and foo_mut methods for each variant? Plus point 1 would require things like size_of and align_of and max to be const functions that can be used in creating an array of the proper size along with a way to set the alignment (current proposals for that require an attribute and you can't use const functions in attributes). So all that complication with a built in macro and lots of language features just to avoid being able to call them a native Rust feature (even though having that auto-generated accessor method syntax extension would technically be a native Rust feature) or having more convenient access syntax? You must really hate FFI users don't you?

So with point 2 you propose having a built in macro/syntax extension which creates foo and foo_mut methods for each variant?

This seems unnecessary. Those accessory methods can easily be created with a user macro or even by copy-pasting the same one line accessor once for every variant.

Plus point 1 would require things like size_of and align_of and max to be const functions that can be used in creating an array of the proper size along with a way to set the alignment

This is necessary for FFI anyway.

(current proposals for that require an attribute and you can't use const functions in attributes)

Could also work with  HKT or even impl Trait:
extern "intrinsic" {
    const fn opaque_union<T>() -> impl Any;
}

struct MyUnion {
    _data: type_of(opaque_union::<(T1, T2, T3)>()),
}

So all that complication with a built in macro and lots of language features

Features that are necessary anyway.

@joshtriplett

Today, Rust has extensive FFI features, but you don't need to pay attention to them unless you build or work with code using them.

I'm curious what you are thinking of exactly. It seems to me that most of the features we have for FFI are kind of "modifiers" on "native Rust" concepts, but I could well be overlooking something. For example, the extern keyword has a role for "native Rust" (extern crate). It can also be used to declare extern fn (but a fn is again something in "native Rust"). The closest thing I can think of off the top of my head are the extern "abi" { ... } blocks -- though those are kind of a shorthand for a bunch of extern items, in practice.
Anyway, I recognize that I seem to be in the minority here in preferring to avoid a "third kind of item" (though both @nrc and @graydon previously seem to have had a similar opinion). Along those lines, I do wonder -- is there a reason we can't use union Foo { ... } as a contextual keyword? (Specifically, when followed by an identifier.) I can't think of one off hand, but I know I am not very good at detecting grammatical ambiguities in my head. (Also, I am not saying I am in favor of this, just exploring possibilities.)
(I know that @tsion raised the question previously.)

@retep998
I'm afraid, point 1 has to be a language feature, so my hate to FFI users can't be fully realized :(
unsafe enum E {
    V1(Type1),
    V2(Type2),
}

E gives you a proper ABI, that's all, everything else (matching, etc.) is prohibited for unsafe enums.
Having proper ABI you can generate accessor methods performing casting from *const/mut E to *const/mut Type1 or *const/mut Type2 manually or authomatically with a derive-like syntax extension or improved bindgen. This is the least invasive change to the language enabling this niche feature.

@petrochenkov The way I've expected point 1 to be done is const fn size_of and macros.

@eddyb Don't forget a way to get the maximum of a set of sizes, so some sort of const fn max that is variadic.

E gives you a proper ABI, that's all, everything else (matching, field access, etc.) is prohibited for unsafe enums.

This could be even less invasive:
#[repc(c_union)]
enum E {
    V1(Type1),
    V2(Type2),
}

Which gives you c union layout. Otherwise you cannot do anything with an object of this type. The only thing that has to be added is this single attribute.

@mahkoh @petrochenkov
unsafe enum E {
    V1(Type1),
    V2(Type2),
}

#[repc(c_union)]
enum E {
    V1(Type1),
    V2(Type2),
}

Take either of those two things, add simple unsafe field access, and suddenly you'd have a solution that makes every FFI user's life so much easier when working with unions.

@retep998 You don't need anything variadic other than a macro which produces binary calls.

I don't think built-in easy access is necessary. It's already possible to get easy access manually with a macro that generates both the type and accessor methods.

@mahkoh If only I didn't have to type both foo and foo_mut for each variant due to the lack of usable ident concatenation. Oh, and initialization, how would that work?

@retep998

Add usable ident concatenation
mem::uninit, mem::init, pick your poison

@mahkoh So provide a third set of variant methods to initialize an instance of the union from that value? unsafe fn new_foo(x: Foo) -> Bar { let y = mem::zeroed(); *y.foo_mut() = x; y }
Also, adding usable ident concatenation has been postponed so much. At the moment the most optimistic forecast is that we'll get it with the new macro system which isn't coming any time soon.

So provide a third set of variant methods to initialize an instance of the union from that value?

I don't see why. The macro that generates the accessor method can also generate
fn new() -> Self { unsafe { mem::uninit() } }

A lot of the time when people want to initialize a union they actually want to initialize it to one of the variants. Right now it is rather unintuitive in that they first have to create it unsafely using mem::uninitialized or mem::zeroed and then assign the value they want using foo_mut. It's about as ugly as having to cast a bunch of raw pointers around or doing transmutes just to get the variant you want. Your new method is rather useless as it can be replaced with a simple mem::uninitialized and so saves basically nothing.

Tbqf, I'd be fine with writing
union! {
    MyUnion;
    T1, t1, t1_mut;
    T2, t2, t2_mut;
    T3, t3, t3_mut;
}

Your new method is rather useless as it can be replaced with a simple mem::uninitialized and so saves basically nothing.

It literally saves up to 15 keystrokes. In any case, I actually discovered the real (TM) reason that those accessor methods are bad:
*my_union.t1_mut() = v;

Does this run the destructor of T1, if any? What if my_union doesn't contain a T1? etc.

Right now it is rather unintuitive in that they first have to create it unsafely using mem::uninitialized or mem::zeroed and then assign the value they want using foo_mut.

And in the case where the field foo: Foo has a destructor:
unsafe {
    let u: Union = std::mem::uninitialized();
    std::ptr::write(u.foo_mut(), Foo::new());
}

This RFC's syntax for the above is let u = Union { foo: Foo::new() };.

@retep998
I've edited my previous message, let a = E::V1(value); was intended to work.

Does this run the destructor of T1, if any? What if my_union doesn't contain a T1? etc.

In general, switching from one variant to another, if both have destructors, will involve drop_in_place (or ptr::read to cause a drop) followed by ptr::write. Maybe this suggests having explicit setter convenience methods, although they still couldn't handle calling the previous destructor without some kind of tag information.

These are all reasons why I think that unions should be a part of the type system, instead of hacked on later. They are filled with footguns, and if we can make it even a little easier to use them, FFI becomes that much nicer :)

I'll write out the basic ideas behind my "the least invasive" alternative.

If code involving unsafe enum compiles, then it behaves exactly the same way as with safe enum (modulo ABI differences).

If code involving unsafe enum would behave differently from safe enum, then it doesn't compile. Basically every code path involving reading the discriminant results in compile time error.

So, unsafe enum's functionality is a strict subset of safe enum functionality (modulo ABI differences), they can use the same internal representation in the compiler.

Super simple and convenient field access to unsafe enums would encourage their use outside of FFI for "optimization" and transmuting. My goal is to avoid this as much as possible.

Despite all of this unsafe enums are still open for potential unsafe-enum-specific extensions, like field access (e.field or e.Variant.field in case of field name ambiguity), but I'd like to avoid doing this in the near future.

Super simple and convenient field access to unsafe enums would encourage their use outside of FFI for "optimization" and transmuting. My goal is to avoid this as much as possible.

Where do people get the nonsensical idea from that it's ok if unsafe code is complex and inconvenient to read and write?

As a follow-up to @nikomatsakis 's note from earlier (https://github.com/rust-lang/rfcs/pull/1444#issuecomment-172903815 ):
My personal preference, currently, would be for a slight variation on @petrochenkov 's suggestion (https://github.com/rust-lang/rfcs/pull/1444#issuecomment-173357432) and the @mahkoh suggestion (https://github.com/rust-lang/rfcs/pull/1444#issuecomment-173358940), at least in terms of the surface syntax for the union declaration itself. It combines the two suggestions, which may seem redundant, but my claim is that they are each adding features that we can treat independently.
#[repr(C,union)] // new; slight variation on `repr(c_union)`
unsafe enum E { // also new
    V1(TypeZ),
    V2 { a: TypeA, b: TypeB },
}

Then you use it like so:
let u = E::V1(SomeZ);
let v = E::V2 { a: ArgA, b: ArgB };
unsafe {
    let the_z = u.V1.0; // this is enabled by `unsafe enum`
    let E::V2 { a: _, b: the_b } = v; // also enabled by `unsafe enum`
    ...
}

So, that provides the convenient accessor-based syntax for variant-projection: enum_val.V1.0, which AFAICT is the main argument I have understood for the #[repr(union)] struct ... suggestion.
My thinking for the semantics of these two additions goes like this:

Let unsafe enum be the introduction form that enables the enum_val.V1.0 and enum_val.V2.b variant-projecting accessors (which, crucially, would only be usable within unsafe blocks).
If you just use unsafe enum on its own, with no repr tag, then the discriminant is still present
That means the language team could decide to put in checked semantics for these variant-projecting accessors on non-repr(union), analogous to how we handle integer-overflow which no checks by default in optimize-mode builds but do include checks by default in debug-mode builds)
The unsafe enum form also allows one to use let E::V2 { ... } = enum_val; as an irrefutable pattern within an unsafe block, as a syntactic variation on the variant-projecting accessor -- again, it might well employed checked semantics, as described above.
All of the variant-projecting accessors treat the input like any other l-value; so they will happily move out of the enum_val if that makes sense, in terms of how the borrow checker treats it.
On top of that, the repr(union) would be the way one specifies that there is no discriminant.
This repr would only be available for unsafe enum instances
The use of this repr would disable the use of the enum as the form for match. (But instead you can use let E::V2 { ... } = ... irreffutably within an unsafe block, as noted above in discussion of unsafe enum.)

As you can see, the example above combines the two repr's (repr(C) and repr(union)) for FFI usage. (This leaves the door open for code making use of #[repr(union)] without repr(C), which may be of interest in some domains where one wants to trust the compiler's choice of layout but still use programmer-based reasoning about which variant is currently installed.)

The use of this repr [repr(union)] would disable the use of the enum as the form for match

This was too strong a statement.
I should have said: disable use of the (now absent) discriminant of the enum for match.
Instances of the enum should of course still be usable with match, to enable things like
#[repr(C,union)]
unsafe enum E {
    Int { tag: i32, payload: i32 },
    Sym { tag: i32, payload: *const char }
}
const INT_TAG: i32 = 0;
const SYM_TAG: i32 = 1;

fn code() {
  ...
  unsafe {
    ...
    match e_val {
        Int(INT_TAG, payload) =>  { ... }
        Sym(SYM_TAG, payload) => { ... }
    }
  }
}

the point being that since E uses repr(union), each variant is now treated as a magically irrefutable pattern (but we can still match on the substructure, which will now have a defined layout due to the use of repr(C) in addition to repr(union)... boy I hope I put those fields in the right place...)

@pnkfelix

Permitting field access on enums with discriminant.

I actually had the same idea! But I rejected it immediately on the basis "Why on earth would you want this?". Maybe it was a rash decision.
It seems like it may be a dynamic (and worse) alternative to variant types:
enum E {
    NumPair { a: u8, b: u8 }
    OneString { s: String }
}

// With variant types
fn process_one_string(os: &E::OneString) {
    println!("{}", os.s);
}

// With enum fields
fn process_one_string(e: &E) {
    println!("{}", e.OneString.s);
}

Let unsafe enum be the introduction form that enables the enum_val.V1.0 and enum_val.V2.b variant-projecting accessors (which, crucially, would only be usable within unsafe blocks).

Why would I ever declare an enum without the unsafe modifier?

@mahkoh

Why would I ever declare an enum without the unsafe modifier?

I'm sorry, I must be misunderstanding your question. Enum without the unsafe modifier is the same enum we all know and love today.  That has lot of use cases.
So I must be misunderstanding.  I will attempt to clarify:
Do you mean "why would I ever declare a #[repr(union)] enum  without the unsafe modifier? (In which case, my answer is "you cannot")
Or do you mean "why would I ever declare an unsafe enum without the #[repr(union)]? (In which case, my answer is "To get convenient field projection, for code where invariants outside of the tag allow you to "know" which variant it is without using match or if let." (the main use case I can imagine for this as a way to prototype code, with useful debug checks, before turning on repr(union).)
Or do you mean something else?

I'm sorry, I must be misunderstanding your question. Enum without the unsafe modifier is the same enum we all know and love today. That has lot of use cases.

unsafe adds strictly more functionality to an enum. And this functionality is not more unsafe than other unsafe functionality such as unsafe functions: It is restricted to unsafe blocks.
Hence, there is no advantage to ever declaring an enum without the unsafe tag.

This is different from unsafe trait which restrict adds restrictions to how the trait can be used. unsafe enum has no additional restrictions.

unsafe adds strictly more functionality to an enum. And this functionality is not more unsafe than other unsafe functionality such as unsafe functions: It is restricted to unsafe blocks.

Ah now I understand your question.
I want to make the variant projection e_val.variant unavailble for enums that are not declared as unsafe enum, because I want to restrict the mental effort necessary when reviewing an unsafe block
If we made all enums implicitly support the variant-projection syntax, then I would need to think harder for every expression of the form e.id that I see in an unsafe block.  By restricting it to unsafe enum, I have a chance of narrowing that set of receiver expressions to a relatively small core that I might be able to keep in my head.

@pnkfelix
Could you clarify, if field access e.Variant.field is used on enum with discriminant 1) Does it panic if the discriminant is incorrect? 2) Does it require unsafe block?
3) Could you give an example of useful code, where the discriminant presents, does not match the specified Variant and the access shouldn't panic?

@petrochenkov my intention regarding enum_val.Variant.field, if used on an enum with a discriminant (i.e., an enum without repr(union)), was as follows:

Does it panic if the discriminant is incorrect?

(Note: this answer is not fundamental to the proposal, but I think it is a sensible extension thereof):
Yes, I would like such panicking to be available as a compiler-injected check for debug-builds; or otherwise turned via an appropriate compiler switch. My intention is that this would be somewhat analogous to how we handle integer overflow: in debug builds (or if you ask for it), the compiler injects the checks that panic, but you otherwise do not pay for it.

Does it require unsafe block

(This is fundamental to the proposal) Yes, it does require an unsafe block. The discriminant is not guaranteed to be checked for correctness, so we cannot in general allow projections outside of unsafe blocks without risking UB.
But (again out of analogy with integer overflow), if the actual variant has a representation that could be mem::transmute to the expected variant, then I would expect that if we do not panic, then you would get the same effect as if you had done the transmute.  I.e., not arbitrary UB.

Could you give an example of useful code, where the discriminant presents, does not match the specified Variant and the access shouldn't panic?

"Useful code"?  No, I think all such scenarios would end up being strawmen on my part; probably such code is more readable by projecting to the properly matching variant and then doing a mem::transmute, since that more directly conveys the effect you are describing, IMO.
So: Why am I suggesting that we allow a non-panicking projection at all, even if only in an unsafe block?  Because I don't want people to pay for the variant-projection feature in release builds, even if they don't opt into using #[repr(union)]. If they've set things up to ensure that this particular variant of an unsafe enum is always present at this point in the code (the same way they would have to if they were instead using a union with no discriminant), then why pay for the check.
(I realize that such checks are likely to be absurdly low cost. But the injected code to perform the panic is higher cost. This is again analogous to the integer overflow issue.)

@pnkfelix

If they've set things up to ensure that this particular variant of an unsafe enum is always present at this point in the code (the same way they would have to if they were instead using a union with no discriminant), then why pay for the check.

If they set things up closely enough to the field access, e.g. in the same function, then LLVM can likely figure this out and elide the checks.
In general, an analogy with indexed access to arrays comes to mind:
e.Variant.field can be re-syntaxed as e[Variant].field and array[index] is normally 1) safe 2) checked (panics when out of range) and 3) the checks can often be elided by LLVM.
In your proposal, the syntax e.Variant.field would be used roughly in the same cases as get_unchecked on arrays, e.g. rarely and under heavy scrutiny. But this syntax is too damn nice to waste it like this!

@mahkoh:

Where do people get the nonsensical idea from that it's ok if unsafe code is complex and inconvenient to read and write?

Moderator note: While I understand the substance of your argument, please remember to be kind and courteous, as required by the code of conduct.  This includes not belittling arguments that you disagree with (“the nonsensical idea”) and not putting words into people’s mouths.
We may delete comments that step over this line (and invite the commenter to re-phrase and re-post them). Feel free to contact the moderators privately if you have questions about moderation policy or about specific comments.

Saying that it's ok for constructs to be complex as long as it's just unsafe code; or even saying that such unsafe constructs should be complex in order to discourage the use of them is nonsense.
Unsafe code is the one place where the supposed advantages of rust, improved compile-time checks, are useless. Hence, if anything, unsafe constructs must be especially clear, especially easy to write, and especially easy to read.
Unfortunately the misguided idea of re-educating programmers of unsafe code by making their jobs harder is somewhat widespread among certain people.

Moderator note: One comment deleted.  That was exactly the type of discourse that we just warned against.  Please feel free to explain/promote/defend your own notion of how best to help users of unsafe Rust, but without using words like “nonsense” or “misguided” to describe other contributors or their ideas.  And again, please talk to the mod team or to individual moderators if you disagree or aren’t sure how to interact within these guidelines.

@pnkfelix

Field access on enums with discriminant.

If field access is permitted on usual safe enums (in either checked&safe or unchecked&unsafe form), then unsafe enum's functionality would still be a strict subset of usual enum's functionality and not something completely different, so I'd be happy with this alternative.

We discussed this RFC at our @rust-lang/lang meeting last night. There was a lot of back and forth. I think it's fair to say that the @rust-lang/lang team is divided at the moment on whether it makes sense to introduce a new kind of declaration or to (ab)use #[repr(union)]. There was though general consensus on the following points:

projection is an ergonomically superior way to select a variant;
however they are declared, unions ought to be smooth to use, which speaks against solutions that generate a lot of macro accessors, as that will never be as smooth as foo.bar
there is a reluctance to add "first-class" features that are strictly targeting unsafe/FFI code, but at the same time a recognition that unions are neither structs nor enums, and a reluctance to violate the #[repr] rule of thumb

We did not reach a firm conclusion on the matter of how unions are to be declared, but I think it's same to say there was some weakening of the resistance to a first-class declaration form. I'll add some more thoughts of my own on that in another comment.
Some other points raised (many of which are just repeating things said here on this thread, and in some cases I may be adding some editorial notes):

You can view enums as disjoint unions and the unions proposed here as non-disjoint unions:

in particular, the borrow checker (in some cases) knows and takes advantage of the fact that if an enum is of variant X, it cannot simultaneously be in variant Y, which means if you borrow fields from X and fields from Y, those borrows can never actually happen simultaneously.
however, that does not seem to be as true for unions. I mean, clearly people can write code that accesses data as two distinct variants -- but moreover that code is not necessarily buggy to do so.
it is a bit unclear then how one ought to interpret unsafe enums -- are they enums that ought to be used like normal enums, but the compiler won't enforce it for you, or are they (non-disjoint) unions?

Adding "variant fields" to enums (or foo as Variant downcasts) is all well and good for the proposed "unsafe enums, but it seems to argue that they should be supported for all enums, which we don't necessarily want.
The integration with borrow checker etc argues against #[repr(union)], but also against union!, since they cannot be understood as desugaring to something simpler.

On the matter of declaration forms, I feel like if it's not going to be #[repr(union)], my preference would be to just use a union contextual keyword. I have not yet figured out if we can get away with this though. :) I tried adapting the YACC grammar in the repo, but encountered difficulties around its use of precedence -- I'll try to scrape out some more time to hack on that.
I still dislike union! for the reasons I've previously enumerated, and I think untagged_union is just, well, unwieldy. Basically, if we're going to go and declare as a first-class kind of data structure, then let's do it well.

Thanks for summarizing our discussion, @nikomatsakis. Here's a very brief summary of my personal perspective:

unions are similar to structs, but only if you squint hard (repr(union), borrowck changes, ...)
unions are similar to enums, but only if you squint hard (no match etc)
we want unions, but don't want to admit it (unwilling to introduce union as a first-class concept)

I feel like we shouldn't try to bend structs or enums to make them work like unions -- there's just too much of a difference to make that worthwhile.
So why not just add union and call it a day? The primary reason I've heard is that "union is really about C/FFI, let's not sully Rust with it". But:

There are likely at least a few other pure-Rust uses. @nrc brought up (de)serialization as one. I suspect we'll find more over time.

If we have the feature, we have it -- regardless of how we hide it. Even if it's somewhat niche, if we think it's worth including in Rust, I believe we should make it stand on its own as a first class feature, given the inability to unify it well with existing features. Who does it help to go out of our way to obscure the feature?

Anyway, I have somewhat more nuanced thoughts I'll try to mention later, but just wanted to leave those thoughts based on the lang team discussion we had yesterday.

My own thoughts since the meeting - I still prefer #[repr(union)] struct, but I have softened a lot on just going with union - I think @pnkfelix's point about how borrow checking union fields is not at all like borrow checking struct fields is persuasive. Furthermore, if we do have a new datatype, then it allows us to pick some nice initialiser syntax - the struct initialiser syntax is not great for unions.
E.g.,
union Foo {
    f: i32,
    g: Bar,
}

fn main() {
    let _ = Foo { f: 42 ); // I don't like this, but it does meet the initialisation == declaration principle
    let _ = Foo::f: 42; // Would this have parsing problems with type ascription?
    let _ = Foo::f(42); // Where did these () come from?
}

@aturon so I think that unions will find a place in unsafe code in general, not just in FFI. At minimum they are a convenient way to take control of dtors, but I suspect they will be used for other representation tricks too.
That said, @eddyb had an interesting proposal of using unsafe type Foo as the declaration syntax. This kind of overloads type quite a bit though, but avoids a new keyword, and makes it clear that this is something which is intended to be used from unsafe code.
I think his bigger point was that one could later extend unsafe type Foo the ability to give very precise control over where fields etc are laid out, a la Ada (as discussed earlier by @nagisa).

@nikomatsakis

You can view enums as disjoint unions and the unions proposed here as non-disjoint unions:
in particular, the borrow checker (in some cases) knows and takes advantage of the fact that if an enum is of variant X, it cannot simultaneously be in variant Y, which means if you borrow fields from X and fields from Y, those borrows can never actually happen simultaneously.
however, that does not seem to be as true for unions. I mean, clearly people can write code that accesses data as two distinct variants -- but moreover that code is not necessarily buggy to do so.

Could you elaborate?
Two distinct variants can be (immutably) borrowed simultaneously, at least at compile time (see the example below).
With regards to ownership unions behave exactly like enums (and not like structs at all) - if one variant is immutably borrowed / mutably borrowed / consumed, then the whole union or enum is immutably borrowed / mutably borrowed / consumed respectively. This is one of the main reasons why I consider making unions an extension to enums ideologically superior to bolting #[repr(union)] on structs - ownership is a central concept in Rust after all.
// Example: simultaneous variant borrowing
enum E {
    V1(u8),
    V2(u16),
}

impl E {
    fn v1(&self) -> &u8 {
        if let E::V1(ref a) = *self { a } else { panic!() }
    }
    fn v2(&self) -> &u16 {
        if let E::V2(ref a) = *self { a } else { panic!() }
    }
}

fn main() {
    let e = E::V1(0);
    let a = e.v1(); // Borrow variant 1
    let b = e.v2(); // Borrow variant 2
}

To expand on the unsafe type arbitary data type definition idea, here's some hypothetical syntax (everything after unsafe type is a huge bikeshed subject):
// An union.
unsafe type Foo<A, B, C> {
    a: A @ 0,
    b: B @ 0,
    c: C @ 0
}

// A tagged union.
unsafe type Bar<A, B, C> {
    tag: u8 @ 0,
    a: A @ 1,
    b: B @ 1,
    c: C @ 1
}

// A repr(C) struct (with const fn).
unsafe type Baz<A, B, C> {
    a: A @ 0,
    b: B @ align(size_of::<A>(), align_of::<B>()),
    c: C @ align(align(size_of::<A>(), align_of::<B>()) + size_of::<B>(), align_of::<C>())
}

@nikomatsakis

I think his bigger point was that one could later extend unsafe type Foo the ability to give very precise control over where fields etc are laid out, a la Ada (as discussed earlier by @nagisa).

That sounds potentially interesting, but we should be careful prior to stabilization to have at least some clear path to such a feature -- we wouldn't want to finalize the design on a false promise. (It also sounds like struct could ultimately be seen as sugar over such a feature.)
To be more clear about my perspective in general, I think it's perfectly fine to add new keywords, if we're adding a new concept. Of course, when possible, it's better to generalize existing concepts to accommodate new use cases -- but only if those generalization are reasonably natural and smooth. From the discussion around this RFC, I'm convinced that unions are not actually a natural generalization of today's enums or structs.
In particular, regarding repr(union), I see the ground rules as:

repr should never change behavior (modulo unsafe code). If you have code that compiles with and without a repr attribute, it should have the same behavior.

repr can, on the other hand, impose limitations on how a feature can be used. That is, a given representation might mean that you can't use a feature as flexibly. But it should always subset the uses that are valid without a repr attribute.

While repr(packed) (and proposals elsewhere like repr(thin) or repr(unsized)) all fit these criteria, I believe that repr(union) does not. It fundamentally changes the meaning of field projection -- this isn't just a matter of layout, it's a matter of introducing aliasing between the fields, if you want to look at it as a struct. Moreover, it has borrowck ramifications and changes the constructor syntax. All of this together mean that #[repr(union)] struct has very little to do with struct except for the projection syntax, and doesn't fit readily into the ground rules for repr in my mind.

@nikomatsakis @aturon Glad to see the general sentiment to add unions to the language in some form; not too surprising to see the declaration syntax as the main point of contention.  I agree with the comment that, once added, they'll exist in the language no matter what syntax introduces them.  In response to @nikomatsakis's later comment, I absolutely agree with using a union context-sensitive keyword if possible; that seems like the best possible choice.  All of the other syntaxes to introduce a declaration really seem like workarounds to attempt to preserve backward compatibility, rather than optimal syntax that we'd choose if not otherwise constrained.
If it turns out we can't use union without breaking its use as an identifier, then in order of preference, I'd suggest unsafe union if possible (still a language built-in) or union! otherwise (ergonomic, but does seem like it ought to desugar to something in-language, which it doesn't).
@nikomatsakis @eddyb In terms of extensibility, I find the concept of unsafe type interesting, in that it could allow the in-language definition of macros for many different possible reprs.  However, defining the corresponding language semantics seems far more difficult.  When can you borrow multiple fields?  "When they don't overlap" doesn't address the unsafety of packed?  (Also, as a syntax nit, having to repeat the field types seems wrong.)  At some point, the language semantics seem sufficiently complex to suggest that any such extension would want to form part of the compiler regardless, at which point an extensible in-language syntax seems less important.
In my opinion, I think it makes sense to add first-class unions even if Rust one day gains an extensible type declaration mechanism that could subsume them.  Such a mechanism could also subsume struct, and enum if it had enough expressiveness to handle discriminants, but even in that case, I don't think we'd want to remove struct or enum.

For the reference I have a draft for flexible layouts (related to Ada-like layout specification) laying around I’m not going to submit. Feel free to give it a read if you want.

@aturon

But it should always subset the uses that are valid without a repr attribute.

If you take away pattern matching from unions and define field access to enums as per @pnkfelix's suggestion, then you will have a strict subset. I still don't see how are unions fundamentally different from enums, everything you can do in absence of the discriminant you can do in its presence by just ignoring it.

@petrochenkov We discussed this a bit in the lang team meeting, and the general feeling is that enums - match + checked "downcasting" are basically a different beast than today's enums. It's not clear to me that we want to introduce such a general checked "downcasting" mechanism for use with arbitrary enums.
(BUT: I do agree that if we added such a feature, the repr approach with enum would make sense.)

@aturon I've personally wanted a way in the past to allow my library's C bindings to expose an enum, something like:
#[repr(C)]
pub struct Event {
    tag: EventTag,
    event: EventUnion,
}

#[repr(C)]
pub enum EventTag {
    None = 0,
    Quit,
    EnterForeground,
}

#[repr(C)]
pub union EventUnion {
    quit: {
        timestamp: u32
    },
    enter_foreground: {
        timestamp: u32,
        to_redraw_x: i32,
        to_redraw_y: i32,
        to_redraw_w: u32,
        to_redraw_h: u32,
    }
}

pub enum EventPump { }

#[no_mangle]
pub extern fn poll_event(event_pump: *mut EventPump) -> Event;
#[no_mangle]
pub extern fn new_event_pump() -> *mut EventPump;

And I'm able to use this fairly nicely in C, C++, or anything else that supports C FFI:
typedef struct {
    EventTag tag;
    EventUnion event;
} Event;

typedef enum {
    EventTagNone = 0,
    EventTagQuit,
    EventTagEnterForeground,
} EventTag;

typedef union {
    struct {
        u32 timestamp;
    } quit;
    struct {
        u32 timestamp;
        i32 to_redraw_x;
        i32 to_redraw_y;
        u32 to_redraw_w;
        u32 to_redraw_h;
    } enter_foreground;
} EventUnion;

typedef struct EventPump EventPump;

Event poll_event(EventPump* event_pump);
EventPump* new_event_pump(void);

int main() {
    EventPump* evp;
    bool running = true;
    while(running) {
        Event ev = poll_event(evp);
        switch ev.tag {
            case EventTagEnterForeground: {
                redraw(ev.event.to_redraw_x, ev.event.to_redraw_y,
                       ev.event.to_redraw_w, ev.event.to_redraw_h);
            } break;
            case EventTagQuit: {
                running = false;
            } break;
            default: { }
        }
    }
}

@petrochenkov

Could you elaborate? Two distinct variants can be (immutably) borrowed simultaneously, at least at compile time (see the example below).

Sure. So here is the example you gave:
// Example: simultaneous variant borrowing
enum E {
   V1(u8),
   V2(u16),
}

impl E {
    fn v1(&self) -> &u8 {
        if let E::V1(ref a) = *self { a } else { panic!() }
    }
    fn v2(&self) -> &u16 {
        if let E::V2(ref a) = *self { a } else { panic!() }
    }
}

fn main() {
    let e = E::V1(0);
    let a = e.v1(); // Borrow variant 1
    let b = e.v2(); // Borrow variant 2
}

It's true that if you executed main to the end, you would have borrows from two variants simultaneously. But of course you can't, because e.v2() will panic. This is precisely what the borrow checker takes advantage of (in some corner cases around matches and loops that I can't remember right now): situations that might otherwise look invalid cannot actually occur. (Note: we are planning on doing something similar] for slice patterns in MIR.)
Now compare what happened there to what would happen if you were using a union: in that case, it would be perfectly fine to simultaneously interpreter e as variant 1 and variant 2 (or, at minimum, it wouldn't panic). So the question then is, "is that undefined behavior, or is it legal to have two interpretations at the same time?"
If we were to go with the name unsafe enum, then it would suggest to me that the value e is just like a regular enum, but you the user are responsible for maintaining the choice of which variant to use. In particular, I would think that, if I add the unsafe keyword (of course), every match that worked on a normal enum ought to work on an unsafe enum (where by "work" I mean: passed the borrow checker, not necessarily did the same thing). But the RFC here tightens the rules around borrow checking, because it assumes that multiple variants may be active simultaneously.
Let me give you an example where this is relevant. The following program is accepted by the Nightly compiler, and rightly so (for some reason, it ICEs on stable). I'll put comments inline.
#![allow(dead_code)]

enum Foo {
    A(i32),
    B(u32),
}

fn main() {
    let mut f: Foo = Foo::B(1);

    loop {
        let mut p = &2;

        match f {
            Foo::A(ref a) => { p = a; }
            Foo::B(..) => { }
        }

        match f {
            Foo::A(..) => { }
            Foo::B(ref mut b) => {
                // Here is the interesting point. We now have a borrow
                // in scope (`p`) potentially pointing at THE SAME MEMORY
                // as `b` -- except that we know that it isn't, because
                // for that to be the case, `f` would have to simultaneously
                // be variant A and B. Moreover, we know that `f` cannot have
                // changed variants since `p` was created, because creating `p`
                // "locks" `f` to its current variant until "p" goes out of
                // scope. More precisely: it prevents reassignment to `f`,
                // though not disjoint subparts of `f`. And that is the point
                // here: fields of variant `B` are considered *disjoint* from
                // fields of variant `A`, even though the two variants reside
                // in the same memory.
                println!("{}", p);
                *b += 1;
            }
        }

        break;
    }
}

@joshtriplett

in order of preference, I'd suggest unsafe union if possible.

Interesting thought. If we were to use a keyword, I definitely prefer union or unsafe union.

On the topic of unsafe type, custom layouts, etc, I've written and re-written this message about three times, since each time I gradually refine my conclusion. Let me just lay out some thoughts that are kind of disjointed so I can get on with my life. I think my current feeling is that custom layouts are very promising but in the end we should just add union though. :)

First, I find @nagisa's flexlayout proposal very interesting. I don't know about the specifics of what he wrote (I left various comments in the gist itself), but certainly the idea that we could add something which could easily address unions now, and potentially scale up to much more complex things in the future is quite appealing to me.
I would like to find some syntax though that lets us easily declare a union now but later support arbitrary layouts. I'd prefer not to have to hash out what kinds of layouts one might want before we can agree on a layout of "everything at 0".
I would love to hear whether people who write Ada code really use this custom layout stuff very much. I could imagine that it is a very general feature that winds up being used in a few narrowly tailored ways, and it might make things way simpler overall to just support those ways directly (or via macros, depending on their importance).
Given that one of the major appealing aspects of this idea is that it can cover bitfields and other such extensions, it seems unfortunate to choose the keywords unsafe type to declare such a thing -- after all, bitfields don't have to be (and should not be) unsafe. My thought would be that you would do #[repr(custom)] and then include #[layout] annotations on fields.
However, there still seem to be some differences between a "struct with custom layout" and a "union" -- primary among them is initialization. How do you select the initial variant? For a union, you just specify the value of a single field (or adopt some of the ideas @nrc was spitballing). But for a struct, I'd expect you to continue specifying values for all fields -- we could potentially have some rules about overlapping fields, like you must only specify 1, I suppose, but that won't work if we wanted to scale to generic contexts like @eddyb was proposing (I'm not sure how important that is, though).
Basically seems like there might still be a case for adding union (and then later allowing custom reprs, if we choose, though I suspect it's too niche of a use case in the end if we have unions already).

However, defining the corresponding language semantics seems far more difficult. When can you borrow multiple fields? "When they don't overlap" doesn't address the unsafety of packed?

Yes, I did wonder about that too. Certainly we could adopt a conservative attitude that says "if the fields might overlap, the borrow checker treats them as unions" -- or just say that when using a custom layout, we always assume overlap. Alternatively, we could be forgiving, but then we would want to specify when the user is allowed to have multiple borrows of overlapping memory, and when they are not (which we have to settle anyway of course, see #1447, but anyway).
I think the issue around packed is related but distinct: that was simply that we need fields to have proper alignment and size to safely borrow them. We could certainly take the same approach that borrow fields is just unsafe unless we can validate their layout/alignment.

(Also, as a syntax nit, having to repeat the field types seems wrong.)

I don't know what you mean by this. Perhaps because @eddyb's example was very generic? In any case, I certainly agree that one should be able to write something which looks very much like a normal declaration. For example, something like:
unsafe type Foo {
    x: i32,
    y: u32
}

(Still, as I wrote, I think I have kind of soured on unsafe type as a choice of keywords.)

@eddyb noted to me that C# has some kind of custom layout support similar to Ada. I'm not familiar with this, but here are some docs that I found:

StructLayoutAttribute
FieldOffsetAttribute

When evaluating the feasibility of "context-sensitive keywords" in the parser, such as allowing union or unsafe union without breaking the use of union as an identifier, please also consider the possibility of nesting unions inside of structs.  While I don't want to tie it into this RFC, I do intend to propose a mechanism for "anonymous union" and "anonymous struct" types at some point, and ideally the syntax should remain consistent between those possibilities.
For example, consider the following hypothetical syntax:
struct S {
    t: T,
    union { f1: T1, f2: T2 },
}

In this case, the parser might not easily handle the use of "union" there, since you could also legally have a field named "union".  "unsafe union" may work better there.

Lookahead for a colon easily solves that, unless you forgot the field name? I don't really understand the intended semantics.
On January 26, 2016 8:47:10 AM GMT+11:00, Josh Triplett notifications@github.com wrote:

When evaluating the feasibility of "context-sensitive keywords" in the
parser, such as allowing union or unsafe union without breaking the
use of union as an identifier, please also consider the possibility
of nesting unions inside of structs.  While I don't want to tie it into
this RFC, I do intend to propose a mechanism for "anonymous union" and
"anonymous struct" types at some point, and ideally the syntax should
remain consistent between those possibilities.
For example, consider the following hypothetical syntax:
struct S {
   t: T,
   union { f1: T1, f2: T2 },
}

In this case, the parser might not easily handle the use of "union"
there, since you could also legally have a field named "union".
"unsafe union" may work better there.

Reply to this email directly or view it on GitHub:
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-174680207

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Nvm, I see (pre morning coffee)
On January 26, 2016 8:47:10 AM GMT+11:00, Josh Triplett notifications@github.com wrote:

When evaluating the feasibility of "context-sensitive keywords" in the
parser, such as allowing union or unsafe union without breaking the
use of union as an identifier, please also consider the possibility
of nesting unions inside of structs.  While I don't want to tie it into
this RFC, I do intend to propose a mechanism for "anonymous union" and
"anonymous struct" types at some point, and ideally the syntax should
remain consistent between those possibilities.
For example, consider the following hypothetical syntax:
struct S {
   t: T,
   union { f1: T1, f2: T2 },
}

In this case, the parser might not easily handle the use of "union"
there, since you could also legally have a field named "union".
"unsafe union" may work better there.

Reply to this email directly or view it on GitHub:
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-174680207

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

It's heartening to read my own thoughts expressed with greater eloquence by @aturon and others. :)
I still wonder about the access syntax though. An observation: "it's more ergonomic" seems to be taken as a consideration trumping all others, and is also implicitly equated with "more ergonomic to write". I think both of these assumptions deserve greater scrutiny. Struct field access syntax may be the most ergonomic to write, but something which makes different things look different would be more ergonomic to read. At other times we put greater importance in the fact that code is read many more times than it is written.
The chief alternative to u.component_name.foo which has been considered so far is (u as TypeName).foo, and as I more readily identify with the position of having to read this code than write it I would still prefer the latter, but maybe we can come up with other alternatives?
Some other possibilities I just came up with off the top of my head:

u.as component_name.foo (if components have names)

u.as TypeName.foo (if access is type-based, or if we have anonymous tuple-like union types like union(i32, bool, char))

u.<TypeName>.foo (likewise; by very loose analogy with foo::<TypeName>)

@glaebhoerl Any variant of u.as something.foo does not read clearly at a glance, because the space in the middle makes it seem like two separate groups of tokens, u.as and something.foo.
However, a variant of dot access syntax could potentially work: u.>something.u2.>foo" would distinguish at a glance between union field access and struct field access.  (Other possible spellings could work there, as long as they read unambiguously and seem evocative of field access; for instance, .~or.!`.)  A separate field access operator doesn't make unions any harder to use, and I can't think of a situation in which I'd want to switch between a struct and a union without reviewing the corresponding users.
That idea of a separate field access syntax already appears in the "alternatives" section of the RFC.

I would be okay with using an alternative operator for field access.

Yes, that entered my mind, but I think the fact that as is a keyword, which will be syntax-highlighted, and is also intuitively thought of as a binary operator - connecting the things to its left and right - would mean that the space in the middle wouldn't be as much of an issue. (And the analogy I had in my mind while writing was &foo and &mut foo.) I don't think a cryptic punctuation-only operator would be much of a win however.

@glaebhoerl The problem arises because "as" can't directly touch the tokens on both sides, since it needs some separation from the field name.  But on the other side of the field name, you can directly continue the expression with a ..  Regarding &mut foo, consider what &mut foo.bar means, and compare to the expression u.as fieldname.subfield.  Spaces should not cause you to misread an expression.
For .>, I took inspiration from C's -> operator, along with the notion that > points towards something and seems evocative of "narrowing".  Any operator that can directly touch both the union name and the field name seems fine; that implies symbols.  And reserving a one-character operator just for unions seems wrong, in addition to not evoking "field access", hence the two-symbol operator starting with ..
I personally don't have an opinion either way on whether union field access should use a separate operator; I'll happily leave that question entirely up to the language team.

So, after a length aside, I've finally gotten my "rustypop" project, which is an LR(1) version of the Rust grammar compatible with LALRPOP, working well enough to test some hypotheses. There is a big caveat to these results, though, in that rustypop is not yet actually tested, so the grammar may be totally bogus. But I believe it to be accurate.
In any case, I tried adding a contextual union keyword, and I am happy to report that it seems to generate no conflicts (i.e., it is unambiguous). You can see the change here on the union branch of rustypop.

@nikomatsakis That sounds extremely promising.  This allows "union some_ident { ... }" at the top level, while still allowing "struct union { ... }" and "fn union(...) ..."?
If we can try this in rustc's grammar, and get the same results, then I'll happily change this proposal to use "union" and define what "contextual keyword" means.

Hopefully having union as a contextual keyword should cut down on the bikeshedding over the declaration syntax so this RFC can get that much closer to landing.

I feel like being able to say "struct union" but not "union struct" would
be somewhat confusing.
On Wednesday, 2 March 2016, Peter Atashian notifications@github.com wrote:

Hopefully having union as a contextual keyword should cut down on the
bikeshedding over the declaration syntax so this RFC can get that much
closer to landing.
—
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-191447234.

@ohAitch it'll be a little confusing. I assume there will be a warning "Don't name your structs union"

On Wed, Mar 02, 2016 at 01:39:19PM -0800, Josh Triplett wrote:

@nikomatsakis That sounds extremely promising.  This allows "union some_ident { ... }" at the top level, while still allowing "struct union { ... }" and "fn union(...) ..."?

Yes.

If we can try this in rustc's grammar, and get the same results, then I'll happily change this proposal to use "union" and define what "contextual keyword" means.

Seems good to me.

@joshtriplett can you update RFC to use the union keyword? I think if that change were made, I'd be in favor of promoting to FCP.

@nikomatsakis Will do.

Any thoughts on being able to promote a struct or union to an enum by defining some trait for it?  I suppose that could simply be postponed for another RFC.

@burdges You mean, custom pattern matching?
@glaebhoerl might be interested in this, as he mentioned it recently, on #1546.
I guess I could see a variant as a pattern over a tagged union, where the tag is matched with an integer constant and bindings for fields of a struct which represents that variant in the enum form the "public variant fields" of the pattern (and they would have to be disjoint).
However, just "implementing a trait" might not be enough to have this working. Value&field refinement would probably be better for this (e.g. type Some = Option { tag: 1, union: OptionUnion { some } };)

Any reason the trait in question wouldn't just be Into?
On Saturday, 26 March 2016, Eduard-Mihai Burtescu <notifications@github.com
<javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:

@burdges https://github.com/burdges You mean, custom pattern matching?
@glaebhoerl https://github.com/glaebhoerl might be interested in this,
as he mentioned it recently, on #1546
https://github.com/rust-lang/rfcs/pull/1546.
I guess I could see a variant as a pattern over a tagged union, where the
tag is matched with an integer constant and bindings for fields of a struct
which represents that variant in the enum form the "public variant fields"
of the pattern (and they would have to be disjoint).
However, just "implementing a trait" might not be enough to have this
working. Value&field refinement would probably be better for this (e.g. type
Some = Option { tag: 1, union: OptionUnion { some } };)
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-201912890

@ohAitch Yes, you could have your own conversion from a C-friendly tagged union to a Rust enum, I thought @burdges was talking about pattern-matching specifically.

Ah I see, so some manner of sugar over
match c_value.into() {
Variant1 => foo,
Variant2 => bar,
HasData(a) => return a
}
perhaps having to do with ambiguity resolution?
On Saturday, 26 March 2016, Eduard-Mihai Burtescu notifications@github.com
wrote:

@ohAitch https://github.com/ohAitch Yes, you could have your own
conversion from a C-friendly tagged union to a Rust enum, I thought
@burdges https://github.com/burdges was talking about pattern-matching
specifically.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-201915381

Yes, I meant a trait to implement the tag needed for pattern matching.

The RFC should be explicit that empty unions are a syntax error. It might also be useful to enumerate the difference between a union with one field, a struct with one field, and the unwrapped type.
Should we allow generic unions? Lifetime-generic unions? I assume yes to both, but under similar rules as for structs, we would need to be able to infer all generics from any field (since we infer actual generic parameters from initialisers).

@nrc I don't think we do need to be able to infer all generics from any field. Unions are far closer to enums than structs; See something like Option: you cannot tell what a thing is if it's only None. I think unions should be treated similarly; you'll have to use each one of the generics in at least one of the fields.

Ah yeah, applying the same rules as enums (i.e., generics must be used, some how) and relying on type inference should be fine.

:speech_balloon: This RFC is entering its Final Comment Period! :speech_balloon:

@nrc, @ubsan: You can definitely define a generic union.  I can mention that explicitly.
@retep998 OK, I'll switch to explicitly requiring repr(C) if you want C layout.  As you said, not hard to drop that later based on usage.

Sorry if this points has been brought up before, but I didn't want to read the entire thread.

Unions and traits
A union may have trait implementations, using the same syntax as a struct.

What about impls? These are not mentioned explicitly in the RFC at all. I imagine they should be supported as well? How does accessing self work for impls and trait impls? Same requirements on using unsafe?

self should work like any other function parameter. Taking a union by value or by pointer is safe, the only thing that is unsafe is doing field access on that union. Since impls and trait impls are just taking the union as a parameter and not forcing field access, there shouldn't be anything special about them.

In particular, any union that has an Into as discussed above can then
use it for entirely safe impls of Eq, Debug, etc.
On Friday, 1 April 2016, Peter Atashian notifications@github.com wrote:

self should work like any other function parameter. Taking a union by
value or by pointer is safe, the only thing that is unsafe is doing field
access on that union. Since impls and trait impls are just taking the
union as a parameter and not forcing field access, there shouldn't be
anything special about them.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-204594891

@nrc

The RFC should be explicit that empty unions are a syntax error.

Done.

It might also be useful to enumerate the difference between a union with one field, a struct with one field, and the unwrapped type.

What did you have in mind?

The RFC should be explicit that empty unions are a syntax error.

@nrc Should we allow empty unions for code generation/macro-friendliness reasons? We originally banned empty braced structs, too, but eventually switched to allowing them.

@tsion
But should that behave like an empty braced struct, i.e. a ZST, or should it behave as an empty enum, i.e. Void?

@eternaleye That's a good question. I guess if I take enum Void {} and split it into a tag part and a union part, enum Tag {} union Data {} struct Void { tag: Tag, data: Data } it seems like the tag which has no valid values is the unconstructible part and the data union is just a ZST, so an empty union would be constructible but useless like ().
I'd rather decide based on some examples of code generation or macros involving unions but I don't have any. We can postpone this decision and relax the rule in the future with better motivation like we did with braced structs.

I'm in favor of postponing on deciding what to do with empty unions and just making them illegal for now.

@tsion I don't think that's quite the same thing.  As far as I can tell, at the time of that RFC, Rust already allowed empty structs, but only with the syntax struct Empty;, not struct Empty {};.
If we end up allowing empty unions, we should absolutely allow them with braces.  But I think for now we can prohibit empty unions by any syntax.

Yeah, I guess it's a different question. Let's postpone until someone has a usecase.

The current RFC proposes the ability to match over Unions, like in the RFC's example:
        match u {
            MyUnion { f1: 10 } => { println!("ten"); }
            MyUnion { f2 } => { println!("{}", f2); }
        }

With the current proposal this relies on the programmer to be aware that matching this way over data with a unspecified layout must/should not be done. Else implementing optimizations like the null-pointer optimization on Option could easily silently break existing code. I think it might be a good idea to forbid matching witch does not just bind variables on a union variant having a type with a unspecified layout. Through having this as a warn/error lint might be enough :)
To quote @retep998:

For non repr(C) types, it should definitely be undefined behavior to access a variant other than what the union was initialized with.

Under the condition that it is specified this way, any comparing match containing types with unspecified layout would trigger undefined behavior when matched exhaustively, I think. Thought I'm not sure if I might not have overlooked something.

@retep998 I don't know about "definitely undefined behavior to access a variant other than what the union was initialized with". You might have an invalid value of type T: !repr(C), but I don't think that having an invalid value should result in undefined behavior:
struct First {
    x: u8,
    y: u8,
    z: u8,
}

union Doop {
    first: First,
    int: u64,
}
{
    let doop = Doop { int: 0 };
    doop.first // NOT UB, but an undefined value.
}

What are the semantics of !repr(C) exactly? I'm guessing some guarantees
are made for unsafe code, but I don't otherwise see why Doop { int: 0} is
categorically ineligible for e.g. being folded to a ZST after a
whole-program static analysis shows that no Doop.int reads occur.
On Tuesday, 5 April 2016, Nicole Mazzuca notifications@github.com wrote:

@retep998 https://github.com/retep998 I don't know about "definitely
undefined behavior to access a variant other than what the union was
initialized with". You might have an invalid value of type T: !repr(C), but
I don't think that having an invalid value should result in undefined
behavior:
struct First {
x: u8,
y: u8,
z: u8,
}
union Doop {
first: First,
int: u64,
}
{
let doop = Doop { int: 0 };
doop.first // NOT UB, but an undefined value.
}
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-205991111

@ubsan: But isn't reading padding UB? Initializing doop.first and reading doop.int seems to be UB no matter how you slice it, and I don't think that kind of subtlety on which is initialized is anything that we want to ask the compiler to deal with - especially because proving it would require a whole lot more static analysis than is currently done.

If you are using union to transmute things, then that means you are relying on the layout, which means you need #[repr(C)]. If someone wants to rely on the layout of a struct they need #[repr(C)] so the same should apply to unions.

@eternaleye It's not UB. It results in an undefined value, with which it's very easy to cause UB. However, the creation of an undefined value in and of itself is not undefined behavior; otherwise, the following would be UB
let x: First = std::mem::uninitialized();

@ubsan: Er, that's not the same as what I meant. My point was that given a union with two members, one of which - a - contains padding and the other of which - b - occupies the whole union, if you initialize with a and read out

I think "a whole lot of static analysis" was in reference to determining
whether the written value contains the read value.
Re:

let x: First = std::mem::uninitialized();

that is defined, but reading from it
let x: First = std::mem::uninitialized();
x.x
would in fact be UB
On Wednesday, 6 April 2016, eternaleye notifications@github.com wrote:

@ubsan https://github.com/ubsan: Er, that's not the same as what I
meant. My point was that given a union with two members, one of which - a

contains padding and the other of which - b - occupies the whole union,
if you initialize with a and read out

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-206174058

…which you were not in fact doing. Right, sorry.
On Wednesday, 6 April 2016, Anton Dyudin <antechno777@gmail.com
<javascript:_e(%7B%7D,'cvml','antechno777@gmail.com');>> wrote:

I think "a whole lot of static analysis" was in reference to determining
whether the written value contains the read value.
Re:

let x: First = std::mem::uninitialized();

that is defined, but reading from it
let x: First = std::mem::uninitialized();
x.x
would in fact be UB
On Wednesday, 6 April 2016, eternaleye notifications@github.com wrote:

@ubsan https://github.com/ubsan: Er, that's not the same as what I
meant. My point was that given a union with two members, one of which - a

contains padding and the other of which - b - occupies the whole
union, if you initialize with a and read out

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-206174058

So then what is the status of
union First {
second: u8
third: u8
}
union MyUnion2 {
first: First
int: u8
}
let foo = MyUnion {int: 5}
foo.first.second
On Wednesday, 6 April 2016, Anton Dyudin antechno777@gmail.com wrote:

…which you were not in fact doing. Right, sorry.
On Wednesday, 6 April 2016, Anton Dyudin antechno777@gmail.com wrote:

I think "a whole lot of static analysis" was in reference to determining
whether the written value contains the read value.
Re:

let x: First = std::mem::uninitialized();

that is defined, but reading from it
let x: First = std::mem::uninitialized();
x.x
would in fact be UB
On Wednesday, 6 April 2016, eternaleye notifications@github.com wrote:

@ubsan https://github.com/ubsan: Er, that's not the same as what I
meant. My point was that given a union with two members, one of which -
a - contains padding and the other of which - b - occupies the whole
union, if you initialize with a and read out
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-206174058

@ohAitch: given that the layout of First is unknown (it's not repr(C) even through it's a union) we have no guarantee about it's layout and therefore it should be UB, I guess.
If first would be repr(C) it should be fine.
(the RFC proposes that: ...default, a union uses an unspecified binary layout...)

This is a really nice RFC, and I appreciate the careful specification of the interaction between this feature and many others in the language. Nicely done!
The integration with pattern matching is particularly slick, and will make programming with unions surprisingly ergonomic.
Over time, I've become increasingly convinced that we need this dimension of expressivity, not just for FFI but also for pure Rust code. There are plenty of cases in unsafe code I've written where I'd much rather use a union than the ad hoc transmutes I currently use.
While I think the potential for a more general "fully custom layout" system (as @eddyb and others have mentioned) is very interesting, I agree with @nikomatsakis that even if we had such a thing, we'd likely still want sugar like union for common use-cases. And in any case, I don't think we should block at least experimenting with this feature on a yet-to-be-designed general mechanism.
Finally, a point regarding repr:

In addition, since a union declared without #[repr(C)] uses an unspecified binary layout, code reading fields of such a union or pattern-matching such a union must not read from a field other than the one written to.

I think this is true today, but I expect that over time we will make some basic commitments about the default repr (and in fact, I've heard some regret that repr(C) isn't the default). Even today, people routinely transmute between different default-repr structs, making some of these assumptions already. So I expect that there will eventually be at least some cases where you can reasonably read from one field having written another in the default repr. (And thus, I don't think we should require repr(C) as some have proposed).

@eternaleye They are both reads of uninit memory.

@aturon I agree: we'll likely end up making some guarantees about the default repr in the future.  I'm happy with the idea of making no such guarantees now, seeing how people want to use this, observing in particular cases where people use repr(C) despite not interacting with C at all, and making some of those cases work with the default repr.  repr(C) starts on the "sharply defined" end of the spectrum, allowing no room for clever compiler optimizations; the default repr starts on the other end, allowing arbitrarily clever optimizations.

@aturon

Even today, people routinely transmute between different default-repr structs, making some of these assumptions already.

This de-facto stabilization is the most unfortunate.
The lack of will and resources to implement field reordering (at least pseudo-random, for future-proofing) or removal of trailing padding (this one affects unions!) in time led to the situation where Rust can lose the opportunity to implement them forever.
Also, layout guarantees for Rust code, not involved in FFI, should not be controlled by repr(C), but by native attributes, something like #[linear_field_order] or #[keep_trailing_padding] (again, this one is applicable unions) which would keep other properties of Rust ABI.
So keeping #[repr(C)] on unions by default is harmful in the long term, unless we are certainly giving up on layout optimizations.

@naicode Please be careful about your use of the term UB. This is not undefined behavior, as described, it simply results in an undefined value. Undefined behavior means "we can compile it any way we'd like". However, the following is fully, and obviously, defined:
union First {
    second: u8,
    third: u8
}
union MyUnion2 {
    first: First,
    int: u8
}
{
    let foo = MyUnion {int: 5};
    let mut tmp = foo.first.second;
    {
        let ptr: *mut u8 = &mut tmp;
        std::ptr::write(ptr, 0);
    }
    tmp
}

@ubsan

However, the following is fully defined, despite reading from perhaps uninitialized memory.

From https://doc.rust-lang.org/reference.html#behavior-considered-undefined :

Reads of undef (uninitialized) memory

So if you're correct in saying it could read from uninitialized memory, It does seem undefined

May have been from the beginning the syntax for enums to do otherwise? Why introduce a similar feature?
enum U {
    f1(u32),
    f2(f32),
}

#[test]
fn test() {
    let mut u = U::f1(1);
    if let U::f1(ref mut b1) = u {
        *b1 = 5;
    }
    if let U::f1(temp_for_assert) = u {
        assert_eq!(temp_for_assert, 5);
    }
}

/*
union U {
    f1: u32,
    f2: f32,
}

#[test]
fn test() {
    let mut u = U { f1: 1 };
    unsafe {
        let b1 = &mut u.f1;
    // let b2 = &mut u.f2; // This would produce an error
        *b1 = 5;
    }
    assert_eq!(unsafe { u.f1 }, 5);
}
*/

struct S {
    x: u32,
    y: u32,
}

enum U {
    s(S),
    both(u64),
}

#[test]
fn test() {
    let mut u = U::s(S { x: 1, y: 2 });
    if let U::s(S {x: ref mut bx, y: ref mut by}) = u {
        *bx = 5;
        *by = 10;

        assert_eq!(*bx, 5);
        assert_eq!(*by, 10);
    }
}

/*
struct S {
    x: u32,
    y: u32,
}

union U {
    s: S,
    both: u64,
}

#[test]
fn test() {
    let mut u = U { s: S { x: 1, y: 2 } };
    unsafe {
        let bx = &mut u.s.x;
        // let bboth = &mut u.both; // This would fail
        let by = &mut u.s.y;
        *bx = 5;
        *by = 10;
    }
    assert_eq!(unsafe { u.s.x }, 5);
    assert_eq!(unsafe { u.s.y }, 10);
}
*/

@jmesmon "NOTE: This is a draft document, and may contain serious errors". This is one of them. And, unfortunately, until we get a real memory model we can point people at, this is the best we have.

@ubsan if that one line is an error, and you know it is an error, then we could do better: you could submit a PR to fix that one error instead of dealing with folks like me reading that document and getting the wrong idea.
It seems a bit backwards that we're complaining about people reading the documentation (normally getting folks to read it is a difficult goal to achieve).
Also, that "NOTE:" text doesn't appear anywhere in relation to the reference (which is what I linked to). It only appears in the rustnomicon. If it should be applied to the reference too, please do add it.

@RustPowers I'm not sure what you're trying to show with those examples.  enum doesn't work like union; it has a discriminant, and takes up more space than its largest field (modulo the null-pointer optimization).  A union with repr(C) provides FFI functionality that an enum doesn't, and a union without repr(C) supports Rust implementations of space-efficient data structures.

@jmesmon I didn't realize that that line was also in the reference. Blagh. I know, it is a real issue. I'm working on it, but it feels like I'm fighting a losing battle sometimes :/

@joshtriplett  enum and union are in one place to store data that can be interpreted in different ways. You propose to gain access through the fields, thereby inventing a tagged enums. Why? Rust's enums can be a little bit inefficient: 1 byte for each variant of the enum, but does it really important in our decade? Nobody in their right mind would not be in external API to expose union, they will be inside the implementation, but with ffi union you can work now. This RFC introduces a new entity which is already present in the language.

Afaict, reading from /actually/ uninitialized memory is that this can cause
e.g. a page fault, which can be problematic if you're already in the page
fault handler.
As for whether
union U {a:u8 b: union {u8 u8}}
let c = U {a:1}
c.b.0
is logically uninitialized, this is why I'm requesting clarification
on what is and isn't a "read".

Afaict, reading from /actually/ uninitialized memory is that this can cause e.g. a page fault, which can be problematic if you're already in the page fault handler.

I think you're mistaking reading from uninitialized memory with dereferencing an uninitialized pointer.

https://www.securecoding.cert.org/confluence/display/c/EXP33-C.+Do+not+read+uninitialized+memory

Uninitialized automatic variables or dynamically allocated memory has indeterminate values, which for objects of some types can be a trap representation. Reading such trap representations is undefined behavior (see undefined behavior 10 and undefined behavior 12)

Exceptions
EXP33-C-EX1: Reading uninitialized memory by an lvalue of type unsigned char does not trigger undefined behavior. The unsigned char type is defined to not have a trap representation (see the C Standard, 6.2.6.1, paragraph 3), which allows for moving bytes without knowing if they are initialized. However, on some architectures, such as the Intel Itanium, registers have a bit to indicate whether or not they have been initialized. The C Standard, 6.3.2.1, paragraph 2, allows such implementations to cause a trap for an object that never had its address taken and is stored in a register if such an object is referred to in any way.

Since https://doc.rust-lang.org/std/mem/fn.uninitialized.html treats them
as equivalent, I'm clearly missing something here.
On Thursday, 7 April 2016, jethrogb <notifications@github.com
<javascript:_e(%7B%7D,'cvml','notifications@github.com');>> wrote:

Afaict, reading from /actually/ uninitialized memory is that this can
cause e.g. a page fault, which can be problematic if you're already in the
page fault handler.
I think you're mistaking reading from uninitialized memory with
dereferencing an uninitialized pointer.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/rust-lang/rfcs/pull/1444#issuecomment-207049138

Reading uninitialized memory is undefined, since it possibly constructs a value not satisfying the type constraints, namely it is possible to construct e.g. an invalid reference.
For this reason, LLVM will produce an undef.

authors: RustPowers Ticki aturon burdges cmr comex eddyb erickt eternaleye glaebhoerl graydon jethrogb jmesmon joshtriplett lilred mahkoh mbrubeck nagisa naicode nikomatsakis nrc ohAitch petrochenkov pnkfelix pornel ranma42 retep998 tsion ubsan