We looked into rewriting the parts of the engine that used immediate mode OpenGL 1.1 graphics, somewhere around 2015. We determined that trying to patch such a change into the game was way outside the ability of anyone on the team, and the best we could get would be OpenGL 2.1 under the compatibility profile with some basic shaders. So what we ended up with was a weird chimera hybrid of 2003 immediate mode rendering code (CPU tells the GPU where the triangles are, one vertex at a time, it's really slow) and a 2015 implementation of shaders from like 2006. The amount of context switching the game ends up doing is considerable, trying to support both of these different pipelines on each frame, and the costs add up pretty fast.
At the time we were testing on a couple different setups, notably Intel Iris Pro (MacBook Pro ~2015) and GTX 1060/similar. This was before AMD GPUs really took off again and we didn't have the resources to test on systems other than the ones we used daily. My FPS target upon release of 2.0 in 2017 was 60fps on that MacBook in nearly all PQ levels. Since then, more features and graphics settings were added, decreasing frame rates without as much performance testing. Combine that with hardware/software moving beyond those older graphics modes, and many implementations just being shims over modern APIs (OpenGL on M1 is just a crappy wrapper around Metal), the frame rates never really recovered.
Also mentioned by RandomityGuy is how the engine itself is single-threaded, and the scripting language is really slow. There is A LOT of script code in PQ that executed every frame, and it can take upwards of 5-10ms on a decent setup. We ported a number of features to C++ that were really slow in MBP <2.0 (radar, network physics, and the new team ported moving objects), but even so there is plenty more that was not. Scripting overhead is not super affected by Wine/M1 translation the way rendering was, but it still adds considerable cost to every frame.
Reverse engineering for MBP/PQ was mostly done using Hopper Disassembler ($80) because it was available for macOS and cheap. A couple people used IDA Pro, but licenses for that start at around $10,000 so you probably won't get your hands on a legal copy. Later on in development, other tools like Ghidra (free) and Binary Ninja ($300) became available and we played around with them, but by that point the mod had shipped and most of the development had been scaled back. These days there is also Cutter (uses Ghidra's backend but with rizen as a core?) and angr Management (under heavy development and imo unusable), but since Ghidra is free and legal that's probably what you'd use for this now.
Most of the reversing I did was on straight assembly since Hopper's decompiler was hot trash, but the people using IDA had access to decent decompiled source code that was a lot easier to read. I would recommend a good decompiler if you're starting out, since assembly (and x86 in particular) has lots of weird semantics that are confusing and require you to read lots of manuals to understand what is going on.
The general process used for reversing the game was to look for functions that sounded like they did what we wanted, and then replace them with our own versions. We were lucky that in the mac version of MBG, the function names were still available. This made it pretty easy to search for functions by name, which we could then use to find those same functions in the Windows version. That process was generally done by looking up strings that were used only in our target functions and finding functions on the Windows version that had the same strings. It worked pretty well. Also helpful was finding the VTables used for all the C++ classes, which showed us where all the virtual functions were for each class.
Secondarily, we also had to reverse engineer structures and classes used in the engine. Luckily for us, we found a copy of a very similar version of the Torque engine, and most of the header files lined up with what MBG did. In cases where there were differences, we cursed a lot and basically just guessed as to what each member of a structure did, using clues from the assembly/decompilation for more accuracy. Hopper had extremely garbage support for structures, so this was mostly just throwing numbers into a header and running the game to see what values came out.
If you're interested in reverse engineering and have a decent computer science background, I would recommend looking into beginner-level CTFs like OverTheWire. I ended up getting a career in that industry, aided by all the skills I picked up reverse engineering MBG, and it's a challenging but rewarding space if you're interested. Also the salaries are astronomical in case that matters to you.
Whew this was a bit of a diversion. Hope this clarifies some of the process by which PQ got to its current state. It's hard to teach an old engine new tricks and you can really see in the subpar performance. I'm not sure of any good solution other than someone like RandomityGuy pouring 100s of hours into profiling and optimization, and I don't think he'd particularly want to do that. You might see some gains with an OpenGL wrapping layer, but the overhead of translation is probably not greater than the savings, and it won't do much to improve script performance. It's worth a shot though if your performance is bad, I just don't know any tools for it off-hand that I would recommend.
Is their a hidden Direct X or vulkan backend? AMD RX 570 poor Open GL peformance
