
Stuck due to “knowing too much” - pooriaazimi
http://programmers.stackexchange.com/q/150549/17163
======
geophile
Some lessons in software engineering take a long, long time to sink in. (For
me, at least):

\- Don't generalize too early, which seems to address the original question
pretty directly.

\- Don't optimize too early.

\- Maintainable > Correct > Fast. Maintainable = comprehensible. If it isn't
maintainable, you won't know if it's correct. If it isn't correct, there's no
point in making it fast.

\- Minimize state. Deriving something from scratch every time it's needed
leads to far fewer bugs, and the bugs there are probably far more obvious.
Cache only when you need to for performance.

\- Simplify as much as you can. This is the hardest one, because it can take a
very long time to figure out all the ways in which a design that you thought
was simple, turns out not to be.

\- Simplify some more: The usual statement of the "simplify" rule is "as
simple as possible, but no simpler". I don't think that's right. If you see a
nice simplification at odds with requirements, push back on requirements.

~~~
mbenjaminsmith
I'm starting to see this more clearly as time passes. I've spent years
developing the ability to "engineer" things and only now am I realizing that
I've learned to "over-engineer" things. Writing software has taken on a life
of its own that is actually counter to its purpose.

I don't think in terms of simplification though, I see it as "write what you
need, when you need it." But I absolutely agree that you should avoid
generalization aggressively and optimization should rarely be on the table. I
also agree about minimizing state. Do it all "on the fly" unless there's known
performance hit.

------
frankc
The moderation on programmers.stackexchange.com is completely overbearing. How
can a question about analysis paralysis be closed for off-topic? If
stackoverflow is for technical question/answer, what is the point the linked
stackexchange if not for discussion of a common programmer problem like this?
I see stuff like this closed all time time and it's really annoying.

~~~
spolsky
This is a rambling discussion, which Stack Exchange is not really intended
for. There are lots of good places to have rambling discussions. We focus on
answerable questions with specific answers, not long-winded, open-ended
conversation starters. There are plenty of other sites for long-winded, open-
ended conversation starters, including Reddit and Hacker News.

After a decade or so of operating sites for programmers on the Internet, I've
learned that, left by themselves, ALL programmer online sites will rapidly
deteriorate into:

* I hate my job, woe is me ("I hate my job too!")

* I'm not productive any more, oh dear ("Oh my god you took the words right out of my mouth!")

* And some nonsense about H1B visas.

These kinds of conversations are endless because, I'm sad to say, they
represent a common problem among programmers. Discussing it as a means of
therapy is a wonderful thing, but DO IT SOMEWHERE ELSE, because Stack Exchange
is 100% focused on questions with specific factual answers.

This hard and fast rule is actually WHY you love stack overflow so much. It's
WHY the network works. It's WHY you click on the stackoverflow.com link in the
Google results even if it's third. If we weren't strict about this stuff we
wouldn't have such consistently high quality ANSWERS.

Just because "moderators" (actually, users) have deemed to close a question on
Stack Exchange, doesn't mean it's a bad question, or that the person who asked
it is bad. It just means that Stack Exchange is an edited, curated
environment, not your livejournal. The very moderation that people complain
about every time a popular question is closed is what has been making the site
work so well.

~~~
fennecfoxen
If there's a meta.stackoverflow.com, have you considered some sort of
socialization.stackoverflow.com? Then you can not only say "buzz off" but
specifically give someone a place to buzz off to. It'd be one of those
pragmatic concessions that keeps the idealism moving along and whatnot :)

~~~
tikhonj
That's what the chat rooms are for[1].

[1]: [http://blog.stackoverflow.com/2010/04/do-trilogy-sites-
need-...](http://blog.stackoverflow.com/2010/04/do-trilogy-sites-need-a-third-
place/)

I _really_ like the "third place" idea, but I don't think the chat rooms have
been successful in realizing it. Unfortunately, people just don't use them
very much--even the StackOverflow rooms rarely have anybody to talk to.

I think the fundamental problem is that discussion in other parts of the site
is asynchronous, like email, while the chat rooms aren't. The comments and
posts stick around and allow people to talk about something across a longer
period of time, which also allows more people to join the discussion. On the
other hand, chat happens immediately--it only works for the people who are
there at the moment and only works while everybody is on.

There is, of course, nothing wrong with chat in general. I just don't think
it's the best match for the SE format.

~~~
JohnMcG
Speaking for myself, clicking on a "chat" link crosses a threshold for me from
"I'm taking a break and checking stuff out" to "I'm goofing off." Maybe I'm
kidding myself that my checking out SE threads isn't goofing off, but once I
enter something called "chat" there's no way I can suspend my disbelief, so I
don't do it.

~~~
yrizos
Heh ;)

So, you're goofing off on chat, a discussion oriented medium, and not on Stack
Exchange, a platform that's feverishly against discussions.

But on more than one occasion you've asked for Stack Exchange to change it's
stance on discussion oriented questions and become more discussion friendly.
Assuming that happens, how soon do you think it'll be before you dismiss Stack
Exchange as goofing off?

~~~
JohnMcG
It's all about plausible deniability....

Seriously, I don't think I would dismiss the whole of SE as goofing off if its
scope boundaries were relaxed a bit.

------
joshklein
This is one of the most pernicious forms of procrastination, because it's hard
for smart people to accept that perfectionism is a negative trait. The more
self-worth you derive from your previous successful outcomes, the more
susceptible your ego becomes to fear of future failures. Your amygdala is
telling you that you are successful already, so don't shake the boat. Tread
carefully and you'll always have an excuse for why things don't work out.
You'll never have to admit you just couldn't do it.

The solution is to divorce your self-worth from outcomes. Instead, derive
self-worth from the process. We can have complete control over our actions,
but never complete control over our outcomes.

This is far easier said than done.

~~~
vibrunazo
Arrogance is probably the trait that most limits our intelligence. I've seen
many smart people do extremely stupid stuff just out of fear of admitting they
are prone to mistakes and failure. Humbleness is hard to master, but it's very
important for us to try and practice it as much as possible.

Potential of success = (Intelligence / arrogance) + luck

~~~
tent1
What you're saying really strikes true, though I feel arrogant people can also
be extremely hard working which can put them in better stead compared to
others. Also, humbleness with a tint of arrogance or self belief can be a good
mix – as long as you get the mix right.

------
hoopism
This is a real thing for sure. The first 80% of a project is the fun/creative
part. The last 20% is painful and usually where you make a product out of a
project. Experienced developers know that starting a project means that
ultimately you will have to do that 20% and they question whether it's worth
the investment (before starting the fun part). New developers are usually
blissfully ignorant and simply enjoy the first 80%. Some of them will take it
beyond that... most will leave it as nothing more than an unfinished project.

The fondness of directing others is really just the enjoyment of not dealing
with the minutia of developing a robust solution.

------
MortenK
Very sad that the moderators are closing such interesting questions, because
they do not conform to their opinion of what should go on the site, even
though the topic is obviously hugely popular and appealing to a lot of users.

~~~
droithomme
It's particularly outrageous because the given reason is "off topic for
software development". It's quite clear from that judgment that the moderators
don't know much about the topic and therefore are not qualified to hold a
position of moderator on a board devoted to software development topics.

------
Aqua_Geek
I also found myself getting stuck and over-engineering. Now, I try to take a
phased approach to development:

1\. Make it work.

2\. Make it good (visually or otherwise).

3\. Make it fast.

~~~
chmike
This good. But how do you handle it when designing a protocol and a data
encoding ? You can't iterate.

~~~
wpietri
All good design comes from iteration. You can certainly do with protocols and
encodings. Look at the very first published swing at HTTP, for example:

[http://www.w3.org/History/19921103-hypertext/hypertext/WWW/P...](http://www.w3.org/History/19921103-hypertext/hypertext/WWW/Protocols/HTTP.html)

From today's modern perspective, it's laughable. And that was just the first
public bit. Surely he iterated some on his own before putting that out.

------
pooriaazimi
I submitted it because it's exactly how I feel lately, and would like to know
what others think about it. My current (personal/academic) project involves
Node.js, (Iced)CoffeeScript, MongoDB (with Mangoose), PhantomJS, Mocha, Neo4j,
Nutch, Solr, Hadoop and Mahout.

All of them are new technologies and are changing rapidly and daily, and I
find new stuff every day that forces me to change my APIs and the way I do
things, and it's getting frustrating... It's the price you pay by using
cutting-edge technologies and I accept that, but I can't help thinking that
maybe Java+MySQL was a better/easier choice!

~~~
gouranga
I suffer from the same trouble on occasions. I primarily deal with the
Microsoft ecosystem which is volatile and fragmented at the best of times.

I would kill to do something in good old java/mysql.

A lot of the new ideas and principles are a mess. Don't get too absorbed -
just use what lets you get the job done quickly and efficiently. a lot of it
is just noise.

I tend to find that a complete break from everything for a day helps. That and
cider :)

~~~
Morg
Try postgreSQL next time, at least it's ACID compliant ;)

------
cheald
Don't let "perfect" be the enemy of "good enough".

A flawed-but-functional system that ships is empirically better than a perfect
system that never ships.

~~~
TazeTSchnitzel
See: Linux vs GNU Hurd

------
wpietri
My solution to this is a simple kanban board. Trello.com is a good on-line
one, but when possible I'll make them out of index cards or post-it notes on
the wall.

People think that a kanban board is a to-do list, but there's a subtle
difference. The real heart of is limiting the number of work-in-progress
items. That means that the backlog is less a to-do list and more a things-I-
am-definitely-not-doing-now list.

For me, every time I have one of those "It would be nice" thoughts, I write it
down on a card for the kanban, throw it in the backlog, and forget it until
the next time I make a prioritization pass. Most of the cards end up in what I
think of as "the pit of nice ideas", the bottom segment of the board that
never moves up because more important things always come up.

For the curious, you can see a photo and a little more explanation here:

[http://www.quora.com/What-are-some-uncommon-ways-to-work-
sma...](http://www.quora.com/What-are-some-uncommon-ways-to-work-smarter-
instead-of-harder/answer/William-Pietri)

~~~
swalsh
Bloody genius!, I'm going to steal some post-it notes from the cabinet right
now!

~~~
wpietri
Good luck! Feel free to contact me with questions.

My big starter tip is to make the work-in-progress column physically small. My
current one limits the "working" and "on hold" columns to 3 items each.
Whenever they fill up that's a sign that there's a problem in my process.

------
stcredzero
A situation I have seen with well-educated novice developers, is that they
think of "technical debt" as universally bad. That's not true. It's not even
true for actual debt.

Debt is only bad if it's your own long-term debt. One can also use short-term
debt as a powerful tool.

For example, it might be simpler for one to write an initial iteration of a
program that's _badly factored._ Then, once you get the program running
correctly, refactor it. It's much easier to know you're on the right path if
your program runs correctly, and you have verification data-sets or unit
tests.

EDIT: Changed one short to long.

------
wpietri
Modestly off topic: Can I say how much I have grown to hate StackOverflow and
its cousins? Just like this article, half of my favorite threads on there are
closed or have been deleted due to violating some sort of rule that is not
unreasonable in theory but leads to obvious idiocy in practice.

A great example of people losing track of the forest because of all the damned
trees in the way.

------
rguzman
this is easier said than done, but the key to avoiding analysis paralysis is
to worry about building the right abstractions and only that. if the software
is abstracted well -- ie it correctly represents the underlying problem at an
adequate resolution -- then it becomes easy to say YAGNI in the short run with
the confidence that refactoring or adding functionality will be straight
forward later.

~~~
Morg
not really. because you can over analyze what the right abstractions can be.
it's only a matter of how abstract you want it to be, believe me.

Analysis Paralysis is inevitable as long as you want to write the RIGHT thing.

As soon as you accept "good enough, we'll upgrade later if necessary", you
start moving towards the right thing.

Anything that says perfect, adequate, right, correct, etc. is wrong and leads
to the dark side.

------
Tloewald
It just struck me we need "slackexchange" where we can trade expertise on
avoiding work.

~~~
yrizos
Dude, we already have one: <http://workplace.stackexchange.com/> ;P

------
h84ru3a
The ideal to strive for is what we call the "Goldielocks effect".

Not knowing too little. Not knowing too much. Knowing just enough to get the
job done.

In computing there will always be multiple solutions, and tradeoffs. What you
want to aim for is "The simplest solution possible."

In my experience, this is really hard for most developers. Maybe the smart
thing to do is find someone who makes it look easy and trust their judgment.
Again, my experience is that most developers are reluctant to do this.

The question is not how much someone knows, it's whether they know "enough" to
get the job done. Goldielocks.

------
powertower
Solution: Don't be purfect. Drop your fears. Enjoy yourself.

