
Expertise Is 'Just' Pattern Matching - TimTheTinker
https://commoncog.com/blog/expertise-is-just-pattern-matching/
======
beat
There are also layers of pattern matching, particularly in creative work. Take
for example bluegrass guitar improvisation, something I've been working at
intensely lately. In order to play a single note, there are several pattern
matches going on at once. First, there's the original song melody and harmonic
structure. The notes of the improvisation need to reflect that base - the
distance the improvisation extends from the melody/harmony without breaking it
is an emotional quality. And this is a moving target! That melody goes by
FAST... bluegrass is often played at tempos exceeding 200bpm.

So on one level, I'm following the melody/harmony. On another level, I'm
inventing (or remembering?) substitutions that can work over that melody and
harmony. On another level, I'm thinking about the overall flow of the
improvisation, and how to achieve my artistic goals, express the feelings and
ideas I have at the moment. On yet another level, I'm being purely technical -
like making my hand slightly roll to play a downstroke on the E string
followed by an upstroke on the B string without accidentally hitting the E
string twice. On another level, I'm listening to the other musicians, with
whom I'm generating a shared tempo, and who are also improvising, generating
new ideas I can respond to while playing.

That's expertise. Being able to work on all this patterns simultaneously, some
being totally orthogonal to others (like "roll my hand" vs "play a sixth
instead of a fifth to imply relative minor").

~~~
rewgs
Hence the importance of "feel" and "flow" \-- they abstract away the
simultaneous complexity.

~~~
beat
That's also the importance of consistent patterns. Bluegrass and bop music,
both extremely high-speed improvisational forms, have a couple of key things
in common. First, time signature... they are strictly 4/4, which makes
"chunking" a lot easier. Second, they are built on really common harmonic
structures (I/IV/V chords in bluegrass, ii/V/I cycles in bop). Third, they
have standard repertoires, and all musicians beyond beginner level are
expected to know a lot of standards by heart.

So the complexity of certain parts of the music are tightly constrained, which
frees the performers to tackle other forms of complexity (like improvisation)
much more aggressively.

------
RcouF1uZ4gsC
> I continue to struggle with the notion that expertise is merely pattern-
> recognition. It seems too simple — there has to be more.

I think people struggle with that because almost all cognition is at some
level pattern recognition. Identifying your mom's face as a baby is pattern
recognition. A dog learning to sit is pattern recognition. A mathematician
finding the integral of some really difficult function is pattern recognition.
A chess grandmaster playing is pattern recognition.

So I think this statement is correct, but it is pretty close to a tautology
and hence not that helpful.

~~~
solotronics
I have a ton of trouble recognizing peoples faces or names, especially out of
the usual context. But I can remember programs I wrote in detail 10 years ago,
network protocols, RFCs/IEEE standards. I think I have decent recall and
pattern matching in some areas at a great expense of some other things. It
took me until I was in my 20s to know what order the months are in a year.
Another anomaly, my hobby besides computers is cars and I can identify every
make and model and most years of almost every car on Earth. The name of my
neighbors? Not so much. Brains are weird.

~~~
thereisnospork
ime a lot of the ease of pattern recognition and recall comes down to
interest. It's likewise easy for me to recall how and why a 302 differs from a
351 or a 460 but I couldn't tell you which letters proceed/follow 'j' without
sounding it out in my head.

My solution has just been to fake it, if I can convince myself e.g. that my
neighbors' names and faces are interesting/important than remembering them
becomes almost impossible not to.

------
skmurphy
I think it's more complicated than this. I think the article is using two
terms interchangeably that have very different meaning:

\+ pattern matching - matching a series of events or a system state against an
existing inventory of models or patterns to find a good fit and take action
based on it

\+ pattern recognition - decoding a pattern at work in a series of events or a
system that is novel to you (or may be a hybrid of familiar and novel
patterns)

As an example of the latter: the fire captain in Klein's "Sources of Power"
who finds himself fighting what appears to be a small fire that is very hot
and cannot be extinguished even after they douse if with water several times.

He becomes alarmed and orders his crew to pull back just as the floor
collapses due to a fire in the basement below. It was not a pattern he
recognized (one that matched prior experience) but it was anomalous in a way
that he intuited was dangerous.

~~~
kedean
I think you can argue these are sides of the same coin. Seeing the new pattern
emerge happens because you are repeatedly seeing the same event over again by
searching prior experiences, it just happens that the experiences were quite
recent. It's pattern matching against recent memories of trying water on the
fire already and seeing that it failed each time.

------
thedirt0115
Being an expert is certainly not "just pattern matching" for physical sports,
music, real-time video games, and many, many other things. Not only does one
need to see the pattern, but they must be able to physically make their body
do what they recognize needs to be done. Beyond "just pattern matching", if
there's any chance of failure, what separates experts from amateurs is
consistency of execution.

~~~
strictnein
I would say there's quite a bit of pattern matching in video games, especially
real-time ones like competitive FPSs.

You see a player round a corner and know that they are likely to end up at
point A, B, or C within times X, Y, or Z.

You hear a specific gun being used to your left, it is likely being fired from
areas D, E, or F because those have good sightlines to where you are.

For both circumstances you know that of the Q number of weapons you have R
will be the best option for the range that you will likely encounter the enemy
at and grenade S will also help. Weapons T and U will leave you undermatched.
You switch to Q, reload it in an area that you know is relatively safe. As
your approach the target you know that enabling powerup V will help you get
the upper hand.

~~~
thedirt0115
I didn't say there wasn't pattern matching in video games, I said that pattern
matching is _only part_ of being an expert.

To use your FPS example, knowing all of those things still doesn't matter if
you miss every shot.

Another example -- 99% of the best Olympic athletes at the 2012 Olympics were
less than 40 years old.[1] I don't think that people over 40 are worse at
pattern matching, I think being an expert at something requires more than just
pattern matching. Depending on the activity, physical strength, eyesight,
reaction time, and so many other things matter too.

[1]:
[https://rua.ua.es/dspace/bitstream/10045/61889/1/jhse_Vol_11...](https://rua.ua.es/dspace/bitstream/10045/61889/1/jhse_Vol_11_N_1_31-41.pdf)

------
joshspankit
Here’s a (potentially frightening, depending on your leaning) thought:

 _If_ expertise is just pattern matching, and AI is almost entirely optimized
for/shooting towards pattern matching, are we to close to AI making experts
obsolete?

~~~
ken
There's many levels of pattern matching, and computers have to start at the
bottom. First you have to convert the real world into a form you can use, like
vision or voice recognition. Then you've got to parse that into concepts. Then
you've got to analyze _those_ concepts for patterns.

Most of our software is still struggling at stage 1, and our software is so
poor at pattern matching it can't detect obvious stage 1 failures at stage 2
or stage 3. We can make a program to recognize species of birds from photos,
but it will also tell us that a bunch of static is (with 99.5% certainty!) a
robin.

I don't think any of the experts I know are at any risk of being replaced by
AI in their lifetimes.

~~~
joshspankit
I don't know if I agree with you there. A lot of AI work is around taking data
and finding relationships between them. A current-gen AI might not understand
what red is, for example, but it has no problem grouping red, blue, and green
in to the the same cloud then mapping the relationships to objects that can be
those colours.

Personall it feels like the current limit on the way to an AI that can take
unknown information and turn it in to a pattern matching system is how much
data it can fit in it's matrix at once.

If I remember correctly (and my info isn't out of date, it's hard to google)
entity matching in photos is usually done internally with very low resolution
files like 320x320 and lower. It's no wonder that it could easily mistake
static for a robin. Take the right person's glasses off and you'll be lucky if
they can tell you it's a bird.

~~~
d1zzy
> I don't know if I agree with you there. A lot of AI work is around taking
> data and finding relationships between them. A current-gen AI might not
> understand what red is, for example, but it has no problem grouping red,
> blue, and green in to the the same cloud then mapping the relationships to
> objects that can be those colours.

It's also worth pointing out that it doesn't matter what red is. All it
matters is that the entity you talk to understand to recognize in the
situations you'd expect them to.

In other words, red isn't some absolute concept, for all we know every human
may subjectively perceive colors differently but most of us can still
recognize the color red when someone asks us to.

------
franze
I am doing SEO since before there was a word for it. 2004ish (Starting from
Dev.)

By now it is all pattern matching for me. I get 1h with a real decision maker
within the company and my way forward is clear. I wrote about it here
[https://t.co/833yGyRDXM](https://t.co/833yGyRDXM)

I sometimes get imposter syndrome, but then I recognize it as a pattern and
get over it.

------
BenoitEssiambre
If by "pattern" you mean, "complex hierarchical probabilistic predictive
models" and by "matching" you mean carefully weighting these models against
each other and using them to predict unknown facts and outcomes and to
generate well calibrated error bars over their predicted behaviours under
different circumstances then, yes.

------
tom-thistime
I think there are two different questions here. 1\. Is expertise reducible to
pattern matching? 2\. Is understanding pattern matching sufficient for
building systems that have expertise?

Probably many people here have seen the argument from solid state physicist
Philip Anderson that #1 does not necessarily imply #2.

"The main fallacy in this kind of thinking is that the reductionist hypothesis
does not by any rneans imply a "constructionist" one: The ability to reduce
everything to simple fundamental laws does not imply the ability to start from
those laws and reconstruct the universe", Anderson, link below.

[https://www.tkm.kit.edu/downloads/TKM1_2011_more_is_differen...](https://www.tkm.kit.edu/downloads/TKM1_2011_more_is_different_PWA.pdf)

~~~
intrepidhero
I wasn't familiar and that was a really great read. Thanks for posting it.

It articulates well the fundamental problem that I struggle to explain when I
talk about why I don't think strong AI will be a thing for a very long time,
if ever.

"The constructionist hypothesis breaks down when confronted with the twin
difficulties of scale and complexity."

------
throw7
By the article's own definition, "expertise" as just pattern
matching/recognition is too simplistic... that leaves out the "action" part
mentioned in the article. e.g. an expert violin player better be able to play
damn well to be called an expert. You have to add practice/work to the mix,
not just pattern recognition.

Being an "expert" is great as a goal, but it's not the only be all end all.
I'm reminded of a plaque from schoellkopf power station that said:

to know what to do... wisdom to know how to do it... skill to do the thing as
it should be done... service

------
bonyt
In Daniel Kahneman's _Thinking, Fast and Slow_ , there is an interesting
section on intuition, and specifically the intuition of experts. It says
something very similar, that "intuition is nothing more and nothing less than
recognition." With expertise, one has more things that they can recognize, and
they can do so quickly and almost effortlessly.

------
BigHatLogan
I find posts like this simultaneously heartening and disheartening. Heartening
because it means a lot of behaviors that are billed as "genius" or "high IQ"
or what have you, might just be an individual unconsciously drawing upon
years, maybe even decades, of experience that is layered on top of each other.
The most obvious example of this, for our community, are actually codebases.
The more you work on it, the easier it becomes to debug and figure out issues,
sometimes without even looking at the code. When I was first starting out as a
junior software engineer, I was shocked at how quickly some of the seniors
could figure out bugs. I was upset because I thought that they might just be
smarter than me, from a raw intelligence perspective.

But now, some years later, I've become that person, at least in my current
company. I can debug issues pretty quickly, and my brain has a store of
literally thousands of bugs and snags that have come up over the years, to the
point where if someone says, "This feature isn't returning the right
variables", I can usually point them to the fix without even looking up from
my laptop. It sounds like magic, but really it's just localized expertise.

That's the heartening part--that all it takes is years to do it. The expertise
is available to everyone, perhaps.

The disheartening part is the other side of the coin--what if you've started
late? Does this mean that software engineers who start late, in their 40's or
50's, for example, might never reach the heights of software engineers who
started early? That isn't true--there are examples to the contrary, I hope--
but maybe they have a much steeper hill to climb. Or they have to figure out a
different approach to getting up the hill, maybe combining years of experience
from a different field.

I love essays about these topics: expertise, pattern-matching, your brain
being shaped by what you do, etc. If anyone has any others, I'd love to see
them.

EDIT: Some grammar mistakes.

~~~
d1zzy
As someone that started "programming" very early on (when I was 9 years old)
and noticed over the years the huge advantage I had over my peers in terms of
"thinking as a computer" I can definitely say that experience matters. But
it's most definitely not all that matters, not even the most important thing
after a certain point:

* other people may be faster learner and may forget less

* other people's interest in the subject matter may decrease at a slower pace than mine's

* other people are simply younger and their body can be "fully awake" for more time, have more energy available to learn (instead of dealing with family issues), their brain is healthier/younger

There's also a unintuitive negative effect of having expertise: you easily
miss out the simple explanations/solutions. This makes total sense, the more
things you about a subject matter the more your brain will attempt to explain
behavior through those means. As you become an expert in a subject matter you
deal with more specialized/detailed aspects which results in your brain trying
to think about those to explain situations or find solutions when a "newcomer"
will very easily figure out that it's a simple problem/solution that can be
done.

So don't feel disheartened, there are pros and cons to being at every level of
expertise, it's a tradeoff.

~~~
BigHatLogan
You brought up some really good points. Your second point about interest--
that's a great point. I think interest--and, subsequently, focus and attention
--are force multipliers, especially in a field like programming, where
experience and expertise don't just help you lay more bricks, but actually
increase how many bricks you can lay in the same unit of time as before.

The negative effect is interesting. I've heard it said before that non-experts
often come up with novel solutions in some fields. That might because of what
you pointed out, that the experts are too focused on a certain range of
solutions, so they might be missing out on a range that a "layman" can easily
identify.

------
jjice
I tutor for time management and study skills at my University, and I tell this
to students a lot, especially when they ask about computer science courses in
particular. I had someone ask how to get better at debugging (she was taking
our C course which is known for giving people a hard time). I struggled to
give her an answer besides knowing the right resources, how to properly search
and troubleshoot, and raw practice.

It made me realize how much of software development is just natural to me
because there's years and years of practice under my belt.

Practice makes perfect.

------
TheOtherHobbes
Creativity is not just pattern matching.

The amount of creativity required for expertise in various domains varies from
"some" to "really quite a lot."

~~~
Psyladine
Creativity is absolutely pattern matching. We're not creative as much as we
are marvellous recombinators.

If a creative were asked what their process is, wouldn't they have to honestly
answer, "I don't know yet, but I'll know it when I see it"?

------
m463
People have said for a long time: "Chess is just pattern matching"

------
aSplash0fDerp
Would this also fall along the lines of "all advanced concepts are build off
of basic fundamentals"?

Pattern matching is just plain fun/entertaining, depending on the domain.

------
xiphias2
Human brain is the most advanced pattern matching engine there exists, so the
titlein itself may be true, but not helpful.

