This would have been easy if I just needed to worry about the letters on the board, but I also needed to capture the letters that were called. I also needed to have a good idea of the location of those letters so I could build my own version of the puzzle board. To get to my goal of real-time letter updates, I knew I would need the ability to accurately detect every letter that appeared. Project Approachīefore we start, I want to explain the approach I took as it will affect how we create the dataset. I have included the scripts I used to generate the images on the project Github repository with the rest of the code. This saved a lot of time that would have been required for annotating each image. This lets me also create bounding box data for each image. Since my code was placing the letters on top of a blank puzzle board, I also knew the exact position and size. I checked letter counts across the selection of words to ensure enough presence of the rare letters. I created a list of 5,000 phrases, many of which were actual Wheel of Fortune puzzles. I created some Python scripts to extract letters from images and used those to create my own puzzle boards. I decided to create my own dataset for my use case. Furthermore, the letters are in the same place, which makes it easier to parse images of the game board. Fortunately, the letters we need to detect are consistent. Getting enough screenshots for good training was going to take some time and there was a challenge in getting enough of the rare letters, like Q or Z, to effectively train the model.

The first step was to collect enough images to be able to train a model to detect letters.

This approach allowed me to capture images from the show to use for training and grab images to send for inference to detect letters. This project uses a USB HDMI capture device connected to a Roku express to capture images from the show while it is being aired. Computer vision to the rescue! After a few weeks of training and coding, I am excited to share a project that does exactly that. The correct ones are visible of course, but there is no way to see the guesses that were incorrect.

This post was contributed to the Roboflow blog by Warren Wiens, a marketing strategist with 20+ years experience in technology who is learning about AI in their spare time.Īs a regular fan of Wheel of Fortune, I often find myself wishing that I could see all the letters that have been guessed for a puzzle.

Wheel of fortune board game youtube