
Minecraft diamond challenge leaves AI creators stumped - ColinWright
https://www.bbc.co.uk/news/technology-50720823
======
TheRealPomax
It's a weird thing to somehow think that an AI with only 4 days of training
data should somehow outperform even a human child, which has a decade or more
of imitation learning experience. A human doesn't come to minecraft "knowing
nothing", like the machine learning programs in this challenge. They already
know basically everything there is to know in order to perform task-oriented
goal planning and only need to focus on task discovery.

If anyone had pass the bar this challenge set, they'd have basically solved
the biggest hurdle in general AI, and would probably have received a call from
John Carmack.

~~~
YeGoblynQueenne
That's kind of the point. The trend in machine learning right now is to try
and solve problems "from scratch" by training "end-to-end", which means
without any prior knowledge. For example, DeepMind's last three or so
iterations of their Alpha-X architecture for game playing did not even have
any prior knowledge of the rules of the games they tackled (chess, shoggi, go
and Atari games) and were trained entirely through self-play and without
anything like an opening book of moves etc, like the origional AlphaGo
architecture (or DeepBlue and traditional chess engines).

That is very big in AI right now, to try and remove the need for the "human in
the loop" and learn complex tasks from 0 background knowledge.

Obviously, like you point out, this is very difficult to do and it requires
lots and lots of data and compute (no, more than that. Really lots). As the
article reports, this eliminates any hopes of "democratising" machine learning
and makes advancing the state of the art a game that only large corporations
can play with any hope of winning.

~~~
semi-extrinsic
> The trend in machine learning right now is to try and solve problems "from
> scratch" by training "end-to-end", which means without any prior knowledge.

In a strong sense, this implies ML is not "intelligent" in any conventional
sense of the word. What little we have of understanding of learning and
intelligence in the scope of humans strongly implies that drawing parallels
with solutions to other tasks, building on previous knowledge, is the key to
intelligence.

Perhaps the purest expression of this is in pedagogy of math. Piaget and
Brissiaud are some of the scholars who I think really grokked this. They
brought forward the concept of _compression_ as essential for learning. E.g.
if you have learned but not yet compressed the operation of addition, you will
have a very hard time learning multiplication. After addition has been
compressed such that you don't have to spend any effort doing it, you are
ready to learn multiplication.

Maybe it would be worthwile to explore if there are AI techniques that can
learn simple tasks first, then use that knowledge to solve harder tasks?

~~~
2muchcoffeeman
> In a strong sense, this implies ML is not "intelligent" in any conventional
> sense of the word.

I don’t know why people have so much faith in ML and AI. Considering what a
CPU actually does, shouldn’t the default position be one of skepticism?

It’d be truly amazing if you can make a machine intelligent in the human sense
by merely coming up with a clever algorithm.

~~~
DavidSJ
Or we could flip your question around:

Considering what a neuron actually does, shouldn't the default position be
that the brain is a computer?

~~~
2muchcoffeeman
No, because you’re being reductive. Biological organisms are clearly not like
silicon computers.

Maybe biologist come up with the next big break through. I mean, if you want
to match an organically grown organism ...

~~~
warent
Is it clear that they're not alike? In the end it's all information and data
being processed, whether by silicon or neural cell. What difference does the
medium make to the end result?

------
CydeWeys
Interestingly, when I started playing the game, this task was essentially
impossible for human players without consulting external resources, because
the crafting recipes were _unlisted_ , and you needed to know the specific
recipes for various things like crafting tables, pick-axes, and shovels that
would be nearly impossible to discover by randomly trying to guess recipes.

I don't know if there's a starting tutorial in the game now that explains some
basic crafting recipes to get started with (there better be!), but unless the
AI can understand it, or you're explicitly programming in the recipes as data,
good luck.

~~~
SquareWheel
Minecraft has had a native recipe book for some time. It displays crafting
recipes as you progress through the game.

~~~
usepgp
That feature has only existed for <20% of the games lifetime. It is more than
likely that the person you're replying to played the game before the recipe
book existed.

~~~
SquareWheel
Yes, they said as much. I'm just letting them know that the feature they asked
about is now in the game.

------
Erlich_Bachman
> Entrants were only allowed to use a single graphics processing unit (GPU)
> and four days of training time.

Alphastar SCII bot has been using much more resources and time than this to
train, so maybe there is one of the reasons no entrant has achieved the goal
yet.

> A relatively small Minecraft dataset, with 60 million frames of recorded
> human player data, was also made available to entrants to train their
> systems.

It will be interesting to see if these artificial and somewhat arbitrary
constraints (although I get that the idea is to restrain it to resources that
are somewhat realistically available to a single individual without
organizational backing today) will either cripple this challenge or in the end
yield some innovative results because the entrants will have to devise
algorithms that use much less data and resources than what has been
traditionally required to get SOTA results.

Additionally it is also unclear whether the way humans learn to play this game
is actually using a smaller or a much bigger dataset to learn from. Sure, a
human can learn to play it in 20 minutes, but that's after 9-10 years of other
pretraining of seing, understanding and operating in the 3D physical world
performing various tasks, getting compressed knowledge from other people by
watching and listening to them... Maybe that would be an interesting challenge
- to still constrain the final model to 1 GPU for 1 day, but at least allow
the model to pretrain on arbitrary similar data, if it is not sourced directly
from minecraft or any clones.

~~~
mkagenius
> single graphics processing unit (GPU) and four days of training time.

Maybe there is a reason they had this restriction? I can only think of
allowing the winning AI to be ready for end users whic h generally have a
single GPU?

~~~
CrazyStat
The resources devoted to training don't really translate to the resources
needed to run the trained model. A model trained on a thousand GPUS for a week
might still run in real time on a single GPU once trained.

The restricted training resources are just part of the challenge. They point
out that a human child can learn the necessary steps in minutes by watching
someone else do it, so they wanted to see if anyone could make a computer
learn it with relatively limited resources.

------
y1tan
_Goes to contest rules :[https://gitlab.aicrowd.com/minerl/minerl-
resources/blob/mast...](https://gitlab.aicrowd.com/minerl/minerl-
resources/blob/master/Rules.md) _Search 'source' :

\- must submit source code \- must submit source code \- must submit source
code

There's your problem right there. It's not that 'creators' are stumped. It's
that, given how much it would be worth pitching the same closed source to an
investment group or solution seeker directly, no 'creator' with a sufficiently
advanced 'new' or capable system would ever submit their source code to one of
these competitions. This goes for all competitions that require participants
to submit their source code. It's what you you seek or your backers after-all
which is valued far more than the potential prize you're doling out. Thus,
your business model. The question is always, will someone who can develop such
an advanced system be dumb enough to part ways with their IP for such low
value? I think not.

So, you can pretty much throw any conclusions made from any such competitions
in the trash. This goes for even bigger ones by bigger names. You're going to
get what you'd intelligently expect : a small sampling of the same ol' same
ol' approach. Don't expect any novel submissions. Don't expect any surprises.
So, what's the point of this? I'm speaking about the whole site 'aicrowd' btw
and any other group that organizes a 'submit your code' competition ...

------
krisroadruck
Is it screwed up that after I saw the whole "20 minutes" thing I immediately
fired up Minecraft for the first time in 18 months to make sure I could still
do it that fast? (18 minutes and change in case you are curious)

------
soup10
Title is misleading. Traditional AI with subroutines for each subtask would
have no problem with this.

~~~
johnmoberg
The challenge was to accomplish this _without_ domain knowledge... if you can
do this with GOFAI, please go ahead and shock the world.

~~~
progval
You have to use at least some domain knowledge, else you would be making a
general-purpose artificial intelligence. The rules actually say: "without
relying _heavily_ on human domain knowledge" (emphasis mine).

The rule is actually quite vague, which is not very surprising, as it seems
quite hard to define what domain knowledge is allowed and what isn't without
having lots of loopholes.

~~~
johnmoberg
I agree that is vague. But there are plenty of RL algorithms that learn with
essentially no domain knowledge, e.g., MuZero[1] which doesn't even use a
model. That doesn't make it AGI -- it only masters a single game at a time and
we currently don't know how to transfer that knowledge to other domains.

[1] [https://arxiv.org/abs/1911.08265](https://arxiv.org/abs/1911.08265)

------
marcelsalathe
Direct link to the challenge:
[https://www.aicrowd.com/challenges/neurips-2019-minerl-
compe...](https://www.aicrowd.com/challenges/neurips-2019-minerl-competition)

------
Buttons840
Y'all got any of them AIs for turned based strategy games?

Seriously, I'm starting to think turn based strategy games like Civilization,
etc, will be the last to get any attention. Why? We need good AIs for these
games and they're not yet solved as far as I know. Furthermore, you don't have
to model the human interface by limiting actions per second like with DotA or
StarCraft. Yes, we solved Go, but turn based strategy games on PC are more
complicated. Seems like a worthy area to research.

Does anyone know of any work on games like these?

~~~
catalogia
> _Seriously, I 'm starting to think turn based strategy games like
> Civilization, etc, will be the last to get any attention. Why?_

I don't think the answer is technical. A good AI for a game like Civilization
would have knowledge of human social concepts like honor and vindictiveness,
which are both concepts problematic to pin down. The definitions of concepts
like these are fluid, they change with the times and societies in question.
But that's not to say any possible setting for these concepts is equally valid
when trying to emulate human behavior. Some settings will seem unrealistic,
like a mustache twirling villain or a hyper-rational pointy-eared alien.
Neither make for good AIs if you're shooting for human-like behavior.

Humans can certainly individually craft fictional personalities that seem
realistic; authors do it all the time. But from a gameplay perspective that
tactic falls flat; you end up with a limited set of personalities the player
becomes familiar with and the game consequently loses relay value. In the
Civilization games, every player knows that Gandhi has a short temper and
likes to launch nukes. Rather than crafting individual personalities, how does
a game designer define a function that returns personalities with a realistic
distribution? How can a game designer define such a function if the parameters
aren't truly known by science and the artists who craft individual
personalities are going off wishy-washy metrics like gut instinct and artistic
intuition?

~~~
sullyj3
> if you're shooting for human-like behavior

You're not though, you're just trying to build a bot that can win the game.

~~~
catalogia
Not if you want a fun game. Civilization is a game about role-playing as the
leader of a nation, not merely about sprinting to the finish. The AI is meant
to take part in that roleplaying, to enhance the experience for the player. An
AI that consistently loses but does so in a very human way is better than an
AI that consistently wins but plays like a machine.

(Obviously the ideal is an AI that plays like a human and can hold its own,
but a playstyle similar to humans is definitely more important than ability to
win.)

~~~
Buttons840
I disagree and have a hard time enjoying opponents that I know are cheating to
enable thmselves to compete.

After I learned the basics my own opinion is that Civilization is no longer a
fun game, due to the poor AI.

We can probably agree that finding fun and interesting ways to dumb down an AI
will be a cool area for research and new ideas in the future, but we need
strong AI before we can figure out fun ways to weaken it.

------
tus88
> Finding a diamond in Minecraft takes many steps - from cutting trees, to
> making tools, to exploring caves and actually finding a diamond.

Sounds like a simple state machine. Don't they program objectives into their
agents? Or are they relying purely on training?

~~~
CrazyStat
Purely training. For example, the rules give as an example of something
specifically disallowed telling your agent what a minecraft tree looks like--
it has to learn that.

~~~
tus88
But humans get to RTFM. Why are denying the autonomous agents the same right?
It is this kind of oppression that will lead the machines to rise up and
overthrow the human race.

------
simias
I'm a bit surprised that anybody is surprised by this lack of success, that
seems like a hugely complicated task given the limited amount of time and
resources. The amount of possible scenarios the AI needs to be able to handle
(fall damage, environmental hazards, block types behaving differently,
lighting, day/night cycles and of course enemies) plus the extremely open-
ended gameplay where you can almost always dig in every possible direction
while collecting and crafting potentially hundreds of items seems to make it a
very arduous task.

~~~
cwyers
Yeah. Look at how complex something like AlphaGo is, and then compare the
complexity of Minecraft to Go. They had four days and one GPU. They never had
a chance.

------
Tempest1981
From what I hear, diamonds are located at random, at Y levels 0-15. The
terrain is usually at Y level 50-60. So digging is required. The best strategy
is to dig on levels 12-15, to avoid lava pools which are on 0-11.

Not sure if the AI can toggle the debug menu, which shows the Y level, or if
it can see the Y level during training.

Is this a job well suited to AI, vs a canned algorithm?

~~~
Tempest1981
Visualization of ore probability by depth:
[https://reddit.com/r/dataisbeautiful/comments/efvgve/where_i...](https://reddit.com/r/dataisbeautiful/comments/efvgve/where_is_each_ore_found_in_a_minecraft_world_oc/)

Observations:
[https://reddit.com/r/dataisbeautiful/comments/efvgve/_/fc2p6...](https://reddit.com/r/dataisbeautiful/comments/efvgve/_/fc2p6h3/?context=1)

------
king07828
My google-fu is failing. I see this and copies of this article, but I want to
know more about the I/O available to the AI. Did the AI get full access to the
Minecraft API or were they limited audio, video, keyboard, mouse like a human
player?

~~~
CrazyStat
The documentation [1] has details. For input they get an RGB image (i.e. what
a player would see on the screen, minus the UI) and other relevant information
(e.g. inventory). For output they have to specify from a list of possible
actions that they can take: move forward, move back, craft an item, etc.

[1]
[http://minerl.io/docs/tutorials/index.html](http://minerl.io/docs/tutorials/index.html)

------
atorodius
One thing I was thinking when reading this is that minecraft is visually based
on the real world. A human opening it for the first time immediately knows
that a tree is a tree. Seeing someone else then cutting it down allows the
brain to do a lot of connections to tree-related concepts.

------
narrator
Sounds like the R/L problems they had and solved with Montezuma's Revenge.

------
uj8efdkjfdshf
I take it no one's submitted Baritone [0] yet? Or would that be against the
spirit of the contest?

[0]
[https://github.com/cabaletta/baritone](https://github.com/cabaletta/baritone)

~~~
CrazyStat
That would be against the letter of the challenge, not to mention the spirit.
From the rules [1]:

> The submission must train a machine learning model without relying heavily
> on human domain knowledge. A manually specified policy may not be used as a
> component of this model. Likewise, the reward function may not be changed
> (shaped) based on manually engineered, hard-coded functions of the state.
> For example, though a learned hierarchical controller is permitted, meta-
> controllers may not choose between two policies based on a manually
> specified function of the state, such as whether the agent has a certain
> item in its inventory. Similarly, additional rewards for approaching tree-
> like objects are not permitted, but rewards for encountering novel states
> (“curiosity rewards”) are permitted.

[1] [https://gitlab.aicrowd.com/minerl/minerl-
resources/blob/mast...](https://gitlab.aicrowd.com/minerl/minerl-
resources/blob/master/Rules.md)

------
datameta
That's ~11k hours of training data

------
faissaloo
Minecraft is a game that is played by consulting manuals and learning from
others, I would be astonished if an AI could magically discover how to find
diamonds.

~~~
detaro
Fairly sure a human would "magically" figure it out with the 11 days of
gameplay footage the AI got ;)

