

Chef Part 2 - Performance - matan_a
http://engineering.voxer.com/post/45944103472/chef-part-2-performance

======
druiid
To me this seems kind of like what I was afraid of way-back-when, when
choosing a CM system. I eventually went with Puppet because basically I was
afraid of this kind of thing happening. Because Chef uses Ruby for the
configuration language this seemed to me both an advantage and a huge
disadvantage.

Basically, Chef is giving you enough rope etc. With Puppet at least, you CAN
get into trouble and make your manifests take these kinds of length of time to
compile, but it's relatively difficult. Additionally, I believe the equal of
'searching' in Chef is PuppetDB. In my experience at least, resolving stored
facts from PuppetDB is extremely fast, as in, fast enough that you don't know
it's actually doing much of anything on a database level.

Really I think the take-away from this is be careful what you do with CM. It
could end up making more, not less work for you!

~~~
IceyEC
Well, searching in Chef isn't like querying a DB, it's a full search against
Apache Solr, a nice wrapper for the Lucene search engine and is capable of
various powerful searches.

~~~
kapilvt
There is something very wrong, if you can't search a corpus of 200 documents
of roughly 512kb size each with solr in milliseconds. Especially on something
as simple as a simple field/facet search for node role match. 2-3s per search
sounds like something else is the bottleneck rather than solr.

~~~
Zombieball
+1 for this comment. 2-3s per search is TERRIBLE performance, and disgusting
given the document count / index size.

------
macros
There is a partial_search available with chef server 11 that makes this
problem much less painful. Search returns the entire node object for each
match, which can be a ton of data as the node count rises. With partial_search
you specify just the attributes that you to be returned.

