Experimenter for Iterative Optimization Heuristics (IOHs), built in* C++.
- Documentation: https://iohprofiler.github.io/IOHexperimenter.
- Publication: https://arxiv.org/abs/1810.05281.
- Wiki page: https://iohprofiler.github.io.
IOHexperimenter provides:
- A framework to ease the benchmarking of any iterative optimization heuristic.
- Pseudo-Boolean Optimization (PBO) problem set (25 pseudo-Boolean problems).
- Integration of the well-known Black-black Optimization Benchmarking (BBOB) problem set (24 continuous problems).
- W-model problem sets constructed on OneMax and LeadingOnes.
- Integration of the Tree Decomposition (TD) Mk Landscapes problems.
- Submodular optimization problems, as seen in the GECCO '22 workshop.
- Several benchmark suites from the CEC conference.
- Flexible interface for adding new suites and problems.
- Advanced logging module that takes care of registering the data in a seamless manner.
- Data format is compatible with IOHanalyzer.
- Dynamic BinVal functions (paper)
- Double funnel functions (paper)
Available Problem Suites:
- BBOB (Single Objective Noiseless) (COCO)
- SBOX-COST (COCO)
- StarDiscrepancy
- PBO
- Submodular Graph Problems
- CEC 2013 Special Session and Competition on Niching Methods for Multimodal Function Optimization
- CEC 2022 Special Session and Competition on Single Objective Bound Constrained Numerical Optimization
The complete API documentation, can be found here, as well as a Getting-Started guide. In addition to the documentation, some example projects can be found in the example folder of this repository.
The pip-version of IOHexperimenters python interface is available via pip. A tutorial with python in the form of a jupyter notebook can be found in the example folder of this repository. A Getting-Started guide and the full API documentation can be found here.
If you have any questions, comments or suggestions, please don't hesitate contacting us IOHprofiler@liacs.leidenuniv.nl.
- Jacob de Nobel, Leiden Institute of Advanced Computer Science,
- Furong Ye, Leiden Institute of Advanced Computer Science,
- Diederick Vermetten, Leiden Institute of Advanced Computer Science,
- Hao Wang, Leiden Institute of Advanced Computer Science,
- Carola Doerr, CNRS and Sorbonne University,
- Thomas Bäck, Leiden Institute of Advanced Computer Science,
When using IOHprofiler and parts thereof, please kindly cite this work as
Jacob de Nobel, Furong Ye, Diederick Vermetten, Hao Wang, Carola Doerr and Thomas Bäck, IOHexperimenter: Benchmarking Platform for Iterative Optimization Heuristics, arXiv e-prints:2111.04077, 2021.
@ARTICLE{denobel2024IOHexperimenter,
author={de Nobel, Jacob and Ye, Furong and Vermetten, Diederick and Wang, Hao and Doerr, Carola and Bäck, Thomas},
journal={Evolutionary Computation},
title={IOHexperimenter: Benchmarking Platform for Iterative Optimization Heuristics},
year={2024},
volume={32},
number={3},
pages={205-210},
keywords={Iterative optimization heuristics;benchmarking;algorithm comparison},
doi={10.1162/evco_a_00342}
}