Creating My Very Own Virtual Reality Baseball Simulator

Virtual Reality

I’ve been very excited about the potential of the virtual reality technology that has been coming out with the Oculus Rift, HTC Vive, and PlayStation VR. The technology is still within its infant stages, however what is available for consumers to purchase and try is pretty amazing. I purchased an Oculus Rift CV1 and am quite impressed with the hardware’s capabilities

There have been some novel ideas in content people have created. Maybe I just have an unusual fascination with simulations, but I’ve found the examples which best demonstrate the current platforms’ strengths are flight simulators (such as Eagle Dynamic’s DCS) and racing simulators (such as Project Cars). This is because the real life equivalent is much more accurately recreated: you remain seated within a vehicle while a 360 degree world goes by around you.

Unfortunately I feel there’s also a lack of imagination with many of the applications that are currently available in the online stores for the platforms. The vast majority seem to be just standard 2D or 3D game concepts that have been successful on 2D displays just adapted to be played with a VR headset. When I put on a VR headset I want to experience visual scales that may not be possible in every day life. I think the flight simulator does this well, a VR-ified Mario knock-off does not.

A Baseball Simulator within VR Limitations

One thing I really wanted to try was a sports game. Now some of you may say, why not just go outside and play a sport? Well you can, but what if the weather isn’t cooperating, or you don’t have a dozen friends who happen to be free to play a pick up game… or you’re just lazy? Anyways, unsatisfied with the available options, I thought it’d be interesting to try program my own baseball game. Specifically, a baseball pitching simulator. Most people after I describe it to them respond with, “Oh cool, so you are a batter and swing at pitches?” No, you just pitch… it’s a pitching simulator… I think it’s cool.

Focusing on pitching only instead of batting is partly due to the VR platform itself. First off, the Oculus Rift has touch controllers which sense the motion of your hands in space. You can’t really hold a bat with them (without significant modification at least), but you can emulate a pretty natural feeling throwing motion with them on. Second is the cues a player needs. The resolution is not fantastic with the VR headsets yet, and I want to create a baseball game at a realistic scale. In order for a player to be able to hit anything the speed and size of the ball would have to be cartoonish. Also there’s no haptic feedback you would get with a physical bat and ball either (although a modded baseball bat with oculus touch sensors and some rumble packs strapped on sounds pretty sweet). If you’re pitching, however, you initiate the play, there’s no cue to respond to. And the useful feedback is the visual trajectory of the ball — just don’t throw the controller on accident. And finally, the pitching is so much easier to sense from the controllers. With the Oculus Touch controllers, the sensor is at the end of the lever arm of the motion you’re trying to track — where the ball is. Even if there are gaps or erroneous sensor readings of the arm motion, it’s a long sweeping arc you can likely fill in. For a hitting motion, you’d have to extend a lever arm of a bat 2 feet out from the sensors on the swinging hands. This would just magnify the inaccuracies in detecting collision with the ball. To deal with it you’d either end up with an overly nerfed hitting experience or just a crapshoot if you’d hit the thing or not.


So I proceeded with my dream of a realistic pitching simulator. I downloaded Unity3D as the graphics engine as well as Blender to make the 3D objects to use within the game. Both free for non-commercial use, which is nice. I did a little Roll a Ball demo in Unity that was actually surprisingly satisfying in VR. I was a virtual god… but only for single-colored spheres and planes. I then went through a series of mood swings over a weekend: as I successfully learned how to create one thing, it would just lead me to discover I had to spend 4 hours now figuring out how to incorporate it. At least the C# programming aspect I was able to adapt to easily, and so eventually I threw together this:


All it’d do was when you press a button on the controller it throws the ball with the same initial velocity over the plate to a cartoon cutout batter and catcher. And for grins it would ricochet off in a random direction within the field of play after contacting the cutout. But it looked pretty cool in VR and you could watch the path of the ball which is pitched at 90 mph. I even set up a couple viewing locations so you’d see the catcher and batter perspectives. It comes out ok in the video, but it looks a lot more impressive with the headset on.


Making Something that Looks Like an Actual Game

Now that I proved to myself I can get something to show up in the headset that looked right, what next? For the moment I was just using the built in physics of Unity which work pretty well, especially for already handling the ground collision detection. But this model just assumes a ball traveling through space with no aerodynamic effects, something I’ll need to change if I want to create realistic ball motion and a repertoire of pitches. But for now it’ll do, and I’ll want to incorporate the human-controller interface in some form first before any fancy baseball dynamics.

So next I needed to up my graphics game. I needed to do away with my Fred Flintstone-looking batter if I was going to share any more of this, and I wanted to get to actually throwing something. So I spent a lot of time learning how Blender works. How to make a 3D model, how to apply materials, how to rig a model, and animate one. I spent a huge amount of time making a low-polygon humanoid batter model in Blender (which was nowhere as good as all the YouTube videos I was watching for help):

Only when I was done did I find out about MakeHuman, an open source 3D human model tool which looks pretty damn impressive, and comes already rigged for you to animate as well. They do come naked though, so I had to make some clothes for him. And once the clothes stopped going through his body while animating things were starting to take shape. I created just two animations: an idle waiting stance and a swing I could switch between. To start with I just decided to make him swing at every pitch (and kept my easy ricochet logic for the ball).

Using Oculus Touch Controllers to Throw a Ball

In addition I finally implemented the ability to throw with the Oculus Touch controller (oh yeah, the initial point of this whole thing). To start with I just took the last reading of the Touch sensor velocity at the time of button release. I programmed the ball to be “held” in your hand while you have the trigger pressed down, and it is thrown when you release the trigger as if releasing the ball. This worked reasonably well, however I found that for some reason if the throw is too aggressive the sensor stops reading and just outputs (0,0,0) for velocity. In other words, try to throw really hard and the ball just drops at your feet. So at the moment I just had to throw softly and wait to fix that issue later. I also created a multiplier on the longitudinal (towards the plate) component of the pitch release. Since I was using the full 60’6″ distance of an actual baseball field, and I’m obviously not making a throwing motion of 85 mph.


The response of the Touch controllers is surprisingly accurate once you have all the rotation transformations figured out in Unity. When throwing a ball, it goes in the direction I expect and I was able to throw it near as accurate laterally as I would expect in real life. I was worried before trying this that the resolution of the Touch sensors wouldn’t allow fine adjustment of throwing angles, but I was impressed.

Some Experiments in Visualization

To experiment with the graphics and Unity’s trail renderer function I tried creating some form of trajectory visualization of the ball. A motivation for this was some interest in possibly turning this into an educational game to learn math and physics. The following video shows the thrown ball with its full quadratic path and the projection of that path onto the longitudinal and lateral rectilinear planes. Some of it is just tossing the ball around in the sandbox environment, seeing how the ball paths look.



Going forward, I’ll look more at creating an AI for the batter and improving the throwing.



One thought on “Creating My Very Own Virtual Reality Baseball Simulator

Leave a comment or ask a question: