You can use this toolkit with any script that has some hyperparameters that need to be tuned.
In order to use Milano you need to do the following things:
Prepare your script (or wrap your function in a script) such that it will accept all hyperparameters as command line arguments and print the resulting validation score in some “recognizable” way (i.e. it should print the result after some pattern, e.g. “Validation score: 0.23”). It should be possible to execute your script with
./<script_name>in the backend environment (see below), but there are no other requirements for what your script should do or in what language it should be written.
- Prepare the Python configuration file. Look at the comments in
examples/os2s/cifar10_azkaban.pyor at other provided configs for examples on how to write the configuration file. In general it should specify the following:
- Path to the script that need to be tuned.
- Which parameters to tune and in which ranges. Supported parameter types are
- “range”: this parameter should be sampled uniformly from min to max values.
- “log_range”: this parameter should be sampled “logarithmically” from min to max values. This means that a uniform value will be sampled between [log(min), log(max)] and then it will be exponentiated.
- “values”: this parameter can be one of the supplied values. * Backend through which your script is going to be executed and workers configuration. Make sure that it is possible to execute your script in the backend, for example, you can run it from the environment where Azkaban is launched.
tune.pyscript to tune your hyperparameters and look at the
results.csv(can be changed with
--output_filecmd argument) file for the results. You don’t need to wait for all jobs to finish,
results.csvwill be updated iteratively on the go. Run
python tune.py --helpto see the list of all available configurations. Example command to train toy speech-to-text model with OpenSeq2Seq using Azkaban:
python tune.py --config=examples/os2s/cifar10/cifar10_azkaban.py --verbose 3
For the full list of supported config options see !!TODO!!
All of the SearchAlgorithm’s support
num_evals parameter which specifies how
many jobs you want to run (or equivalently, how many function evaluations you
want to do) and
random_seed. Currently the following SearchAlgorithm’s are
Python >= 3.5 with packages listed in
Python package is required.