Yeah, you just do a single click on "+" (which says "Start chat"), and then it says on top "Start a conversation with someone using their name, email address or username (like @foobar:matrix.org).", so you just type in @username:server. Is it difficult? Could it get any easier? There might be other ways as well, such as the one you mentioned.
