Recently, the AI research company Epoch AI released an interactive simulator designed to simulate the computational power required to train large language models. Through this simulator, researchers found that while a 2012 outdated graphics card (like the GTX580) could be used to train GPT-4, the cost would be ten times that of modern hardware today. Epoch AI's research indicates that the number of floating-point operations (FLOP) required to train GPT-4 is between 1e25 and 1e26.