Need advice: Transitioning from basic ML to building an external RL bot for a 3D game
I want to learn how to build an external AI player that plays a game from the outside (by capturing the desktop screen and simulating WASD keyboard inputs, not by modifying the game files).
The game is a fast-paced 3D game where the player must dodge oncoming obstacles. The control inputs are simple (just WASD), but the game visuals are very chaotic with flashing, shifting background colors.
My Background:
* I have some mid experience with Supervised and Unsupervised Machine Learning.
* I want to build a real RL agent that learns by trial and error using a reward system.
My Questions:
-
Since the background visuals flash and shift colors constantly, raw pixel inputs might take too long to train. Is it a good idea to extract object bounding boxes first, and use those coordinates as the numerical input features (Observation Space) for an RL model?
-
What libraries or frameworks should I look into for wrapping an external Windows desktop window into a custom Gymnasium environment?
-
Are there any specific GitHub repositories, tutorials, or open-source projects could help me?
I am ready to learn and write the code, but I just need to be pointed in the right direction.
submitted by /u/EE83_
[link] [comments]