I am currently writing a forum CMS (http://pygm.us/IbkgNZ4d), and I, too, have wondered how to expand the number of tags within a rigid nomenclature. Using a lexer that automatically converts links to tweets and YouTube videos to embedded scripts seems like an unintuitive solution, but on the other hand, people unfamiliar with the commands will automatically display the embeds in their posts.
For commands that don’t convert links, I was thinking about something like ':<COMMAND>:' with an expandable list of whatever people prefer. Something Awful already use this model for their emoticons, and it could be extended to things like images in general like states for election conversation and such.
I have always found the `[url]` solution to be incredibly bothersome and to some extent unintuitive, and with the advent of mobile devices, letting people write their posts in as few characters as possible is a big advantage to be taken into consideration when weighing the pros and cons.
Lots of vBulletin forums use the `:<string>:` syntax because vBulletin's default smilie set ships with that form of syntax (even though you can use arbitrary strings like setting "lol" to display a laughing gif) so everyone just piles onto it. Basically, direct string replacement seems to be universally understood by all users.
But the real riddle here is devising a syntax superior to bbcode that transcends string replacement and does things like take arguments and act like functions.
Because, it's this less-straightforward symbolism that requires the higher order of savviness/pattern-recognition that less-experienced users struggle with. Like `!()` turning into an image (but not `! ()`) or why you'd need to indent 4 spaces to resume a bullet point after an empty line.
In other words, where you and I may find it obvious that we're conforming to the rules of a parser (on some back-of-the-mind intuition at least), I found that this concept of mechanical recognition is nonobvious to the user archetype that expressed confusion over Markdown. To them, `! ()` doesn't work because of a negative rule "there can't be a space", not because the token is simply no longer recognizable to the robot behind the curtains that renders their post. That's the crux I've arrived at that makes Markdown suboptimal for my particular community demographic.
The final point I discovered is that users almost always use the toolbar button for anything that comes from their clipboard (namely image and website URLs). Click, paste, and done. Even on smartphones. So essentially all Markdown did there was take cumbersome syntax that was seldom typed-in to begin with and replace it with less intuitive syntax for a benefit that was seldom awarded: being easier to type! Users then had to confront the `()` beast when editing posts or modifying their post's layout.
Fun stuff to ponder. It's always extremely eye-opening and humbling to be so wrong.