Thanks to the rise of quantum computers, many variations of the variational quantum eigensolver (VQE) have been proposed in recent times. This is a promising development for real quantum algorithms, as the VQE is a promising algorithm that runs on current quantum hardware. However, the popular method of comparing your algorithm versus a classical baseline in a small basis set is not meaningful in the big picture. Moreover, many papers use a different molecular representation or a different quantum computer to test their algorithms such that the used baselines are different between different papers. Thus, it is almost impossible to compare the different algorithms to each other. As a solution, we have built a benchmarking framework to standardize the VQE performance metrics, such that they can be analyzed more easily. Using our framework, any researcher working on the VQE can easily test their own algorithms against previous ones on the leaderboard without the need to reproduce previous work themselves.