
StateOfTheArt.ai - EvgeniyZh
https://www.stateoftheart.ai/
======
sgibat
Really awesome, definitely bookmarking. Couple points:

\- Shouldn't you link to the arxiv page instead of the PDFs in case there's a
revision?

\- What about a column for "area" as well? Would be nice to scroll through and
see what areas have made progress relative to others lately.

\- Some sort of very brief mini-about/summary at the top of the main page
wouldn't hurt. A tagline of some sort.

\- Now I notice there's an About under the hamburger menu -- why not just
display HOME and ABOUT instead of hiding them in a menu? You don't have enough
menu options to need to hide them in a hamburger menu.

~~~
andreyk
Yeah, hamburger menu is weird. Also, it'd be nice to have a column for code
(just links to github really).

------
nine_k
The point:

 _We are an enthusiastic group of students trying to put in our two cents into
the fast developing fields of Artificial Intelligence (AI) and Machine
Learning (ML). ... One space we believe is currently not covered is that of an
easily consultable repository for state of the art, quantifiable results
across tasks. ... The main source of data is the community that can upload the
relevant results of their field! We have further aggregated data from reading
(a lot) of articles, from scraping the web, and from several existing datasets
and websites. A portion of our dataset came from the Electronic Frontier
Foundation’s "Measuring the Progress of AI Research."_

Kudos for the effort. I wish more areas had something like this.

~~~
pugworthy
If they would put that at the top of the website page, it would help 1000%.

~~~
ACampero
thanks, added a comment

------
alexrigler
Great to see another one of these resources. I'd highly recommend checking out
NLP-progress [1] and Papers with Code [2] which I've found to be excellent
resources.

[1] [https://nlpprogress.com/](https://nlpprogress.com/) [2]
[https://paperswithcode.com/](https://paperswithcode.com/)

------
ACampero
Hey. One of the creators here. Yes we are still missing several tasks and
results, the idea is that this could be maintained by the community by making
it easy to upload results. Definitely help is appreciated We are collecting a
lot of feedback

~~~
codewithcheese
Hey thanks for this. Please add a finance category. Timeseries has some
overlap with finance but there is plenty of finance AI research that is not
restricted to time series. Financial data has many distinct characteristics
and interesting challenges.

------
trott
Commendable effort, but badly outdated already.

For example, the best result on CIFAR-10 for density models that I know of is
2.85 bits per dim (PixelSNAIL) rather than 3.35 (Glow), and this is a year
old. PixelCNN++ is well known, and got 2.92.

For classification on CIFAR-10, current SOTA is much better than the 2015
Fractional-MP result (which also used non-standard augmentation, so perhaps
not directly comparable to the rest)

~~~
ACampero
Yes. the idea is the community can help maintain it by adding results (and
tasks or datasets)

~~~
trott
I think this kind of resource is in a chicken-and-egg situation: for people to
bother to contribute to it, it needs to be seen by others as an authoritative
reference, and vice versa.

I agree that it's badly needed. There were other projects like this, that seem
abandoned:
[http://rodrigob.github.io/are_we_there_yet/build/classificat...](http://rodrigob.github.io/are_we_there_yet/build/classification_datasets_results.html)

This probably belongs on Wikipedia.

------
pugworthy
From the "I'm purposely playing dumb" perspective, I totally don't get that
website on first glance. Even a 6 word sentence summary could make it a lot
more interesting.

------
gok
Looks super cool! There is a bit of data clean up to do. Just looking at
speech recognition: "WER" and "Word Error Rate" should be the same thing, and
sometimes it seems to be on a scale of 0 to 1 and other times a percentage.
Also the Switchboard test set is duplicated. Finally, it really should be
marked when data is augmented; many of these numbers are trained on outside
data, which says more about how much data the researchers have access to as
opposed to the ML system design.

------
garfieldnate
When I think of the things I would add to this, I realize how amazingly useful
it would be to have more data on the tasks themselves. For starters, whether
or not there is code and what license under which it is released, and for NLP
what languages were used/are available for use. Also, sign language
recognition fits somewhere between computer vision and NLP. If picked up by
the community, this could become a huge knowledge base.

------
mortivore
I'm seeing duplicates on there. Maybe find a way to group them, then when you
click the link you can then choose from a particular source?

------
pouta
I'm toying with the idea of making an Open Source repository of reproducible
papers. How did you get this started? How did you find people to collaborate?

I already bought reproduciblepapers.com

------
mendeza
For the state of the art for segmentation on the Cityscapes dataset, PSPNet
isnt there and to my knowledge at 2017, it was state of the art (SOTA). I dont
see it on this site.

~~~
ACampero
Yes. There are some missing results and tasks. The idea is the community will
help maintain it by adding results, tasks and datasets

------
ni-hil
Very cool, will definitely keep this in my favorites.

However there is an issue with the pagination/navigation. I can't see the next
items if there is more than 20.

------
lucidrains
I was just about to build this! Arghh you beat me to it..

~~~
throwawaylolx
I'm sure they'd be happy to accept help!

~~~
ACampero
indeeed!

~~~
lucidrains
what's your email??

------
asutekku
I can’t scroll the list on safari (iPhone) and instead have to use the small
are left to the list scroll. Otherwise really cool site!

------
youeseh
Hey OP! This is really cool. Would there be a way to add a column that
explains the uninitiated what each column means?

------
SubiculumCode
Very cool. I'd have liked a section concerning 3D Medical Image
Segmentation/Classification.

~~~
ACampero
Its easy to add datasets, results and tasks. the idea is the community can
help maintain it :)

------
haldora
I really like that html table you are using! It's featureful and looks great.
:)

------
lousken
can't connect, doesn't use TLSv1.2

------
deepnotderp
For Atari they forgot Rainbow I think

