
Show HN: Exp, a tool/lib to configure, run, and optimize experiments in python - davidenunes
https://github.com/davidenunes/exp
======
davidenunes
I created this tool to design parameter spaces, run functions in parallel and
optimize hyper-parameters for machine learning models, it's pretty generic and
a work in progress, but it gets the job done. There are possibly corner cases
I didn't consider, but I'll add/fix things as I need/find them.

~~~
Immortalin
Is this like a simple version of Google Vizier-style blackbox optimization?

~~~
davidenunes
Yes, I just wanted a way to separate the model logic from the experiment
design. So you design a parameter space in a configuration file, and either
use the optimizer to find the best parameters within that space, or run every
possible combination of parameters (grid seach)

There are plenty of ways to optimize models from evolutionary algorithms to
bandit models, in my simple case I'm working with limited resources, so I
included a wrapper to a global bayesian optimizer for sequential search. Grid
search is out of the question, but the tool is still handy if I want to freeze
all the parameters and vary a couple of them to study some aspect.

bonus: the parameter space file is a nice way to document my experiments --I
can track exactly how each experiment was configured.

