Note the original title is: T* makes for a poor optional<T&>
The last `<T&>` there is key, but seems to have been removed from the HN version of the title.
This blog post is not arguing for the use of optionals in general (it assumes everyone already agrees with that). It is instead making a much more wonky argument that std::optional<T> should support T being a reference type, which apparently is not supported right now.
For most purposes, it seems std::optional<T&> would behave very similarly T*, but the ergonomics would be better in certain template scenarios. It seems like a valid argument to me, but definitely getting into the weeds a bit.
Aside: IMO std::optional in general is disappointing because it doesn't actually solve the worst problem with nullable pointers: dereferencing a null std::optional is still undefined behavior. I wish the standard had gone with a design that forces the developer to write an explicit check when unwrapping the optional...
> Aside: IMO std::optional in general is disappointing because it doesn't actually solve the worst problem with nullable pointers: dereferencing a null std::optional is still undefined behavior. I wish the standard had gone with a design that forces the developer to write an explicit check when unwrapping the optional...
You can use `.value()` to deref an optional instead of `operator*` or `operator->`. It will throw an exception on empties.
IMHO the primary purpose for a function returning a `std::optional` instead of either returning a value on success or throwing an exception on failure is that you can use it in codebases where exceptions are disabled for whatever reason. If the behaviors of `.value()` vs `operator*` were swapped, such that `operator*` throws exceptions and `.value()` is undefined behavior, all the people who wanted it for the purposes of eschewing exceptions, the crowd who wanted `std::optional` the most, would be getting the verbose code.
Which is bad at every level if resolution: it’s more verbose to do “the right thing” (obviously), it’s more expensive to do “the right thing” (because if you’ve properly checked you’re paying the check twice and praying the compiler can remove the redundant check), and it’s incompatible with other pointers to do “the right thing” (because pointers use ! And *).
So .value() it mostly worthless, if it’s being used it’s an assertion with overhead which can take down your program.
> all the people who wanted it for the purposes of eschewing exceptions, the crowd who wanted `std::optional` the most, would be getting the verbose code.
And nothing of value would be lost (except for it not behaving as a “standard pointer” anymore).
> it’s more expensive to do “the right thing” (because if you’ve properly checked you’re paying the check twice and praying the compiler can remove the redundant check)
Eh that's hardly a solid complaint here. Since these are header-defined templates, it's all inlined. And any compiler will absolutely remove those redundant checks as a result. And an unchecked .value() would still be available if you really did object to that "overhead", so it doesn't violate any C++ principals here.
I do kinda think that operator-> & operator* as a result did the assertion, and value() bypassed it. But this would also be the opposite behavior of other std:: classes, like std::vector (operator[] being unchecked vs. at() being checked).
I do not think it is. What does the language do on * dereferencing? Yes, it does not do a check. So if you do it now you have to remember two different rules, one rooted deep inside C and C++ minimal overhead philosophy. Now multiply by the number of types (unique_ptr, shared_ptr, optional and probably more) and remember the correct rule.
I do not think there is something bad with it. It does the same for the same syntax and you have a `.value()` checked access anyway.
Throwing an exception is somewhat better but still not what I really want, usually. I want the compiler to make me write an if()-like statement checking if the value is non-null before I can dereference it.
Try/catch is if-like if you do it right.... But it's also not mandatory. Hence the argument for Java's checked exceptions... which didn't pan out in the end.
Throwing exceptions in C++ is also incredibly slow, you generally don't want to do it in any path that you expect will actually be exercised in normal operation.
Checked exceptions were a nice idea in theory that definitely didn't work well at all in practice. IMO exceptions should be reserved for "panic" situations, as a friendlier alternative to crashing. (But I also feel they should be used liberally in such situations...)
I cut my teeth on Python's "forgiveness not permission" model and find that exceptions are usually the cleanest model since so few languages have pleasant syntax for alternate return-types, and returning a status code and using out parameters for the actual result is miserable.
Sure but python was built on a model of signalling anything via exceptions, C++ was not, and it sacrifices the exceptional case for the benefit of the nom-exceptional one.
Disabling exceptions means that you can't catch them, not that you can't throw them.* If an exception would be thrown under such circumstances, the process aborts. This would have been a safer default than undefined behavior (though inconsistent with other fundamental and standard C++ APIs, which generally prefer for the obvious default to be fast rather than safe).
* This is technically wrong, because, e.g., destructors don't run on the way out; I'm going for pithiness here rather than strict accuracy.
Well, that seems to be using macros to cover up the syntax, which of course the C++ committee isn't going to recommend or use for anything new (nor should they).
The actual implementation of Maybe doesn't seem to force anything, it just returns a pointer to the stored value. There's no enforcement of any kind that anything is checked as a result. There's some hoop-jumping with friend classes to "hide" the readMaybe() function to make the non-macro usage as ugly as possible to strongly encourage using the macro-convention, but that's hardly "enforcement".
It's a cute syntax & macro pattern, but it's hardly a good fit for a C++ standard, either.
The C++ committee could come up with actual syntax for this so it doesn't need macros.
> There's no enforcement of any kind that anything is checked as a result. There's some hoop-jumping with friend classes to "hide" the readMaybe() function to make the non-macro usage as ugly as possible to strongly encourage using the macro-convention, but that's hardly "enforcement".
Sure fine. Anyone who wants to reach into the internal APIs could also just as easily do `#define private public` and bypass every protection C++ has. We're not trying to defend against malicious developers here.
To be fair, languages which require explicit unwrapping also provide syntactic short-hands for the most common idioms. Long-handed if/else (or pattern matching) is used to implement the short-hands or as a last resort.
See also value_or and the C++20 monadic methods on std::optional.
Why, other languages that require unwrapping it still see plenty of people using optional. In fact, I would have more inclination to use it in my projects if that was the only way to do it.
Rust turns an Option<T> into a raw nullable pointer[1] (with NULL being Option::None). This is cool because a 2-item enum with a single pointer would usually be 8 (ptr) + 1 (tag) = 9 (total) ≈ 16 (padded) bytes, but it's only 8 because of this.
I'm simplifying a tad, look up 'null pointer specialization rust' for more. What's cool is the zero-cost abstraction over type layout specification, while allowing the generic Option<T> type to work exactly as expected.
Yes, more of these would be nice, the ones we have are very valuable but it would be nice to make our own, ideally safely, but unsafe ones made carefully in a trustworthy library are good too.
I would also like being able to tell the compiler about other niches. If a Foozle can be Ding, Ping, Zing or Number(n) where I promise n is from 0 to 199, that all fits in a byte if the compiler is willing to work with me by storing Ding, Ping and Zing as magic sentinels in the niche range 200 to 255 and I'm willing to trade the small runtime cost for the memory usage win.
> If a Foozle can be Ding, Ping, Zing or Number(n) where I promise n is from 0 to 199, that all fits in a byte if the compiler is willing to work with me by storing Ding, Ping and Zing as magic sentinels in the niche range 200 to 255 and I'm willing to trade the small runtime cost for the memory usage win.
Yeah, that would be cool. I think it would make sense to represent this as a repr(u8) enum with a range-limited Number variant. Something like:
#[repr(u8)]
enum Foozle {
Ding,
Ping,
Zing,
Number(u8 in 200..=255), // or whatever range syntax
}
go suffers from this problem. Lack of generics makes it impossible to define an optional<T> type, and *T is annoying to use and requires an extra allocation.
In practice, people use a zero value to indicate that a value is missing, but this only works if the zero value isn't also a valid input.
Hopefully the coming generics will resolve this pain point.
Generics will let you define an Option[any], just hope that your code reviewers don't decide that this isn't goey, and you should "keep it simple" or "just use a pointer" or something in that vein.
There won't be overloading * for getting the value inside and overloading bool conversion to check if the value is there. Still, it's an improvement.
One philosophy is that zero input basically never be a valid input unless you're taking in integers. https://www.infoq.com/presentations/Null-References-The-Bill... . The idea is to try and never create null objects. I think it's interesting and helpful in practice to check/fail early.
But many types do have a "natural" zero value, that you may want to use, beyond just the numbers. Consider the empty string, empty list, empty dictionary, empty set. Those are zero values for their types. By reserving them as pseudo-nulls you lose the ability to distinguish between an object which happens to be empty and an object which is meant to mean "null". This is where Optional (or similar mechanisms) come in handy. If you actually want to avoiding creating null, Optional is a cleaner and more consistent way to accomplish this without needing to magic up a null-value in your type's valid set of values or create a one-off Optional (MaybeDictionary, MaybeList).
It's even worse than that in my opinion: the zero value for maps and slices is actually nil even though they're not pointers, meaning the empty values are not the zero values. This lets you do "fun" things like this (my memory of the syntax might be slightly off):
```
var map1 map[int]int = nil
var map2 map[int]int = &map1
var map3 map[int]int = nil
```
Interface types also can be nil even if they're not explicitly pointers because heap pointers are needed under the hood due to the size of the value not being knowable at compile time. You can pretty quickly get into some hairy situations trying to ensure that a value isn't nil when combining pointers or maps/slices with interfaces.
I really wish that separating the concerns of present versus not present from value versus reference was more mainstream. I understand why this was traditionally the case for lower-level languages, but even higher-level ones like Java are mostly based on the paradigm that only references can be "not present". This irks me in a similar way to how earlier versions of Java wouldn't allow default implementations of interface methods and instead requiring abstract classes, which classes could not extend from more than one of.
Yeah, I think this might be a tab/spaces mixing situation like python had, where it had to be restricted by the language compiler because it's just so tempting.
> One philosophy is that zero input basically never be a valid input unless you're taking in integers
This is obviously not possible, since you can imagine lots of complex objects are all integers or themselves composed of integers, and for this entire class of objects, the zero value usually makes sense.
For example, a 3-vector (in the physics sense) is a 3-tuple of integers `type Vec3 struct {x, y, z int}` , `Vec3 {0, 0, 0}` being the origin. How does a function specify that it can take an optional 3-vector? Even worse, you can imagine a struct for a potenitally 0-volume cube represented as 8 Vec3s, one for each vertex, whose 0 value is again a valid object.
Of course, you could gratuitously add an `IsValid bool` flag which must be true to the class, but the cube example shows how this quickly becomes annoying and bloated.
In this case, the memory pointer for Vec3 would not be null? It will be a defined memory address which points to 3 zeros. I think it's a difference of objects and primitive data types.
The discussion was that in Go, given the current lack of generics, you have two options to have an optional parameter:
1. Take a *Vec3, using nil as the "optional is missing" value.
2. Take a Vec3, but consider some value as "invalid", often the 0-value as possible.
1 invokes exactly the issues in Hoare's essay. 2 doesn't work for all types, as some types don't have any bit pattern that are not useful, this is what I was trying to point out.
I can't think of any situations where (1) would be insufficient, but I'm probably wrong. But I agree that setting an isValid bit for every object is overkill.
Well, in the link that you shared, Tony Hoare explains in quite some detail why having support for null pointers is not great.
Additionally, a *Vec3 is more than `Vec3` or `null`. For example, the following code will behave very differently based on whether optionalVec is Vec3 or *Vec3.
optionalVec := Vec3{0, 0, 0}
foo(optionalVec)
fmt.Printf("theVec: %+v", optionalVec)
// would print {0, 0, 0} with an Optional[Vec3] type
// will actually print {2, 0, 0} with *Vec3
...
func foo(optionalVec *Vec3) {
if optionalVec != nil {
optionalVec.x += 2 //we use a different origin
}
[...]
}
I think the point is that it would be nice to be able to express the idea of a value not being present without having to risk accidentally trying to dereference a nil pointer. That's what Option brings to the table, although arguably Go could just provide an implementation of that like they do for slices and maps without actually having to introduce full generics to the language.
I don't know about others, but we chose to use Go despite the problems with the language itself because we liked several properties of the Go runtime. Particularly, the fact that Go is garbage collected (so it's memory safe), but has much lower memory and start time overhead than even the latest versions of Java or .NET or even Haskell; and the ecosystem is much greater than something like OCaml. This low overhead is very useful for a microservices-based application.
For my team, we would have loved to have something like C# (or even Java), with generics, exceptions, lambdas, LINQ/Streams etc., but with Go's low overhead and large ecosystem. Go the language was a major negative point, just not bad enough to outweigh the advantages of the runtime.
> people use a zero value to indicate that a value is missing
Go supports returning multiple results. Maybe I don't read enough Go code to understand what's actually popular, but AFAIU the idiomatic way to accomplish this in Go is something like:
if v, exists := lookup(key); exists {
do(v)
}
or
if r, err != foo(); err != nil {
do(r)
}
The compiler, unfortunately, can't enforce proper conditionals. OTOH, unlike C compilers Go won't make any dangerous assumptions about dereferencing nil pointers, which is something.
I'm not sure Generics can help here. What you really want is compiler enforcement of comprehensive condition checks. For example, via pattern matching switch statements. That's easiest to do with optional types as the unwrapping operation creates a simple point for static constraint checking, but in principle the same thing could be accomplished with annotations on multi-value return types that describe the association.
> Whenever the idea of an optional reference comes up, inevitably somebody will bring up the point that we don’t need to support optional<T&> because we already have in the language a perfectly good optional reference: T*.
But i have never heard this said even once. Rather, inevitably someone will bring up the point that you can do it with reference_wrapper. Check it out:
I don't really use optional in C++, but it strikes me that it would be pretty hard to reason about the lifetime of the T&. Pointers have the same problem, but when you see a pointer you're automatically going to start thinking about ownership, whereas references (especially if auto comes into play) are a lot more subtle...
Also if you're going to optionally store a reference why not just use a unique_ptr or a shared_ptr so ownership is much more obvious and safer. (I realize it might not be zero cost, but optional<T&> just seems like an invitation to hard to find bugs)
Well, I mean look, T* is a poor optional<T> simply because an optional<T> behaves like a T when present, whereas a T* makes no pretense that it's not a T.
If X isn't Y, and doesn't even try to sort of be a Y when you need it to in useful circumstances, then X is a poor optional Y for that excellent reason.
A tail is a poor optional leg, for starters, and so on.
What you need is some_kinda_smart_pointer<T> instead of T*, which can be T* when you need it to, but also quacks like T most of the time; like when passed as an argument to a function needing a T, it will convert to that by yielding the T value and so forth.
The name "some_kinda_smart_pointer" can be "optional", and QED.
A T* isn't a class type; it's a basic type: a pointer. You can't do too many clever things with basic types in C++. They can't have any member functions, so they cannot substitute anywhere where something is required that has member functions.
TLDR: naked pointers inherited from C have disadvantages compared to smart pointers in all sorts of generic programming and whatnot; ask any C++98 programmer.
I think you greatly misunderstood the article. There's no issue with optional<T>, nor is the article complaining that T* is a bad optional<T>.
The problem is that you can't have optional<T&>. So if you want to pass an optional reference to something, you're currently forced to pass T* instead (using null as the proxy for "not passed", of course). And thus, T* makes for a poor optional<T&>.
Didn't say there is (or, for that matter, that there isn't).
The author does say there is an issue with it: he would like optional<T&> but cannot have it. (That, of course, is not optional's fault, though).
> nor is the article complaining that T* is a bad optional<T>.
But it is poor (not just in comparison to a fictitious optional<T &>).
> So if you want to pass an optional reference to something, you're currently forced to pass T* instead (using null as the proxy for "not passed", of course). And thus, T* makes for a poor optional<T&>.
If you want an optional reference to something, maybe you don't entirely understand references. A reference isn't optional; it has to refer to something.
References in C++ are odd; they have limitations. For instance, you can't declare an array of them, either.
To simulate an optional reference, we use a pointer, and we can wrap that in a class:
class optional_int_ref {
const int *maybe_int; // null if we aren't referencing an int, otherwise valid pointer
};
Then of course we want that to be a template to make it generic.
std::optional is a container: it contains a value, which may be indicated as not being present. A reference isn't a value which is contained; it's a value which is somewhere else. If you contain a reference, the reference must be resolved to a valid object.
Basically, you don't want optional<T &>, but a parallel optional_ref<T>: something that optionally references a value, and if so, that value is somewhere else.
So, today, we could make optional_ref<T>. If we did so, then T * would be a poor optional_ref<T> for the same reasons that it's a poor optional_ref<T &> that we don't have. Therefore, T * is not the best optional we can have.
> Basically, you don't want optional<T &>, but a parallel optional_ref<T>: something that optionally references a value, and if so, that value is somewhere else.
This exact consideration is discussed in the article and arguments are brought forth against it. I would love to give you benefit of the doubt, but you clearly didn't read the article.
It's a long article; I read most of it, but skimmed some of it quickly toward the end.
Thanks for pointing that out; see the "§ What about optional_ref<T>?" just before the conclusion (same name and all).
I probably missed it because it has no code box; if I get tired of reading something long, I start to gravitate toward code rather than walls of text.
optional_ref being "spelled differently" is a given. It has different semantics from optional, and doesn't replace it (as is noted) and so of course it deserves a different name. An optional reference to something (which itself isn't optional) isn't the same as an optional something (which either exists or doesn't). This is a fluff objection.
Regarding the objection against duplicating code in the implementation for things like map:
1. So what? C++ is full of duplication.
2. All the same, perhaps the implementor could find a way to share some implementation bits between optional and optional_ref. I haven't tried a cut at it, but it doesn't seem self-evident to me that the only way to implement optional_ref is complete copy and paste from optional.
The author has a valid points about there being issues with having to hard code either optional or optional_ref in places in the user code.
Maybe a combind super_optional<bool ref, T> which goes to optional<T> if ref is false otherwise to optional_ref<T> could help.
That is similar in spirit to the workaround, except the choice of "ref or not" isn't extracted from a T& versus T argument; it's an explicit template parameter.
In the end the author's conclusion is indeed that "optional_ref<T> is a much better optional reference than T*, but it’s still a poor substitute", where by substitute I think he means "alternative for the nonexistent optional<T&>".
#include <optional>
#include <functional>
#include <type_traits>
using std::optional;
using std::conditional_t;
using std::reference_wrapper;
using std::cout;
using std::endl;
template <typename T>
class optref {
T *ptr;
public:
optref<T>() : ptr(nullptr) { }
optref<T>(T &obj) : ptr(&obj) { }
optref<T> &operator =(const T &rhs)
{
if (!ptr)
throw 42;
*ptr = rhs;
return *this;
}
};
template <bool ref, typename T>
using opt = conditional_t<ref, optref<T>, optional<T>>;
int main()
{
int x = 42;
opt<true, int> rox = x;
rox = 3;
cout << x << endl;
}
Compiles with GNU g++ -std=c++17 -Wall -W -pedantic.
The output is
3
showing that rox is a reference to x. Via the hacky assignment operator I threw in for this purpose.
We can build abstractions on opt<ref, T> but they have to carry that pesky parameter. If we can move it to the right and default it to false, that helps if most of our optionals aren't references.
> If you want an optional reference to something, maybe you don't entirely understand references. A reference isn't optional; it has to refer to something.
I'm not sure how you wrote this but then didn't immediately realize why optional<T&> would be useful.
> std::optional is a container: it contains a value, which may be indicated as not being present. A reference isn't a value which is contained; it's a value which is somewhere else. If you contain a reference, the reference must be resolved to a valid object.
And yet std::reference_wrapper exists and basically ruins your entire thesis. There's no conceptual problem with storing references in containers; `struct Foo { int& i; }` is exactly that and has worked since forever. Rather it's a problem with the implementation of optional<> specifically that prevents it. And why reference_wrapper exists, to "fix" those implementation issues - primarily with types that already existed like vector, which is why it's silly it's still needed on new types like optional.
The key is to avoid equivocating between "C++ reference" and "some kind of value that indicates another value elsewhere".
C++ references cannot be optional by design; that is one of the aspects of pointers that C++ references "fix", and you cannot code around it. A key idea behind references is to get rid of the null-based optionality of pointers (nowadays referrred to as the "billion dollar mistake").
The author of the article would like to use the declaration syntax of references T& in the middle of some templateology to produce an optional reference; but that optional reference itself cannot have anything to do with C++ references.
If the abstraction doesn't actually have C++ reference semantics in it, why would you want to specify any aspect of it using C++ reference syntax? Because it is poetically memetic or something?
reference_wrapper doesn't actually have anything to do with references; it doesn't have to be used with references and isn't implemented literally as a wrapper around a reference: implementations contain a pointer. It really should lose the _wrapper from its name, which provides no information about it.
Where is the reference here?
#include <iostream>
#include <utility>
#include <functional>
using std::reference_wrapper;
using std::cout, std::endl;
using std::swap;
int main()
{
int x = 42, y = 73;
reference_wrapper rx(x);
reference_wrapper ry(y);
swap(rx, ry); // swap pointers
cout << rx << ", " << ry << endl; // 73, 42
cout << x << ", " << y << endl; // 42, 73: stayed where they are
}
It looks pretty much like a rudimentary smart pointer.
> C++ references cannot be optional by design; that is one of the aspects of pointers that C++ references "fix", and you cannot code around it.
You keep repeating this as if it's at all relevant. The single purpose of std::optional is to make things optional that cannot themselves be optional by design. That C++ references cannot be optional are exactly why it's a perfect fit for std::optional.
You seem to be talking about the difficulty in storing T& if the value isn't present. That's not actually hard to do, that's what std::reference_wrapper does already. The C++ standard library already solved that, and trivially so.
> reference_wrapper doesn't actually have anything to do with references
What? Literally from the documentation:
"std::reference_wrapper is a class template that wraps a reference in a copyable, assignable object. It is frequently used as a mechanism to store references inside standard containers (like std::vector) which cannot normally hold references."
The only thing it does is take a T& and return a T&. That it stores it internally as a T* is an implementation detail.
As for your example snippet, you just have a bug in usage since you didn't read reference_wrappers docs. It needs to be `swap(rx.get(), ry.get());` since the _wrapper part of its name is actually significant, you have to get the wrapped value out of it. Alternatively `swap((int&) rx, (int&) ry);` would also have worked.
std::optional makes for a poor optional simply because doing the right thing is not the default, requires more typing and uses an interface that doesn't exist in T*.
It seems like an unusual rephrasing of the saying "reasonable on the face", which means "reasonable upon first appearance, without considering deeper aspects."
My understanding (not the author): "prima facie" is a legal term for "on its face", and this is saying the same thing but phrased in a different way. It looks reasonable on its face, due to being "superficially similar" (author's words). But when you look a bit deeper, it is not.
If a developer is committed to using template-fu, then of course T* doesn't fit nicely with all the template magic, and the developer should use a template magic version of T* which is 'optional'. For those who don't like template-fu, T* is way better than 'optional', not simply because 'optional' is a template and therefore bad, but because T* works just fine in this world. So I am confused by the point of this article. It's like saying "gasoline makes a poor diesel when used in my diesel engined car."
The last `<T&>` there is key, but seems to have been removed from the HN version of the title.
This blog post is not arguing for the use of optionals in general (it assumes everyone already agrees with that). It is instead making a much more wonky argument that std::optional<T> should support T being a reference type, which apparently is not supported right now.
For most purposes, it seems std::optional<T&> would behave very similarly T*, but the ergonomics would be better in certain template scenarios. It seems like a valid argument to me, but definitely getting into the weeds a bit.
Aside: IMO std::optional in general is disappointing because it doesn't actually solve the worst problem with nullable pointers: dereferencing a null std::optional is still undefined behavior. I wish the standard had gone with a design that forces the developer to write an explicit check when unwrapping the optional...