March 16, 2019

AR RPG-My lifelong dream

Today marks the 5 year anniversary of my first game ever published to the app store: a rudimentary role playing game made in a simple engine called RPG Maker, you can find it here. To commemorate this occasion, I wanted to do something special. Ever since I was a child, I have always dreamed of owning my own company and creating all sorts of software, from video games, to artificial intelligence, to robots. One such idea I’ve always dreamed of is an augmented reality game. To commemorate this anniversary, I decided to flesh out this idea that has always resided in the corner of my mind, and gauge how costly, feasible, and most importantly enjoyable it would be.

Ever since I first got into video games, I’ve always been fascinated by the idea that one day there might be a game that both takes place in the real world, and a virtually created one. This concept further cemented in the depths of my subconscious with recent TV shows, books, moveis, and games revolving around some concept of this idea. Whether it be Sword Art Online, Ready Player One, or Pokémon Go, during my early middle school years this concept became more and more of a possibility rather than a fantasy.


Image result for ready player one gif oasis                      Image result for pokemon go gifImage result for sword art online gif nerve


Quite safe to say that I am not the only person in the world with dreams of crazy new ways to blend the fictional and real

With my latest Hackathon, I learned much about both the current field of AR and VR, but also how to actually create augmented and virtual environments. I now had at minimum a starting point in this fantastical idea. With this as a launching point, my creativity flourished. After much brainstorming and thought, and many more hours of research of what the current field is like, I have decided on the following base concept. A concept, which as of now has now name. A concept that will surely be my ultimate life goal. One that will take an entire company of like-minded individuals to pursue

The basic idea is simple. A role playing game, that rather than occupying an electronic screen, such as the games of today, as well as my first game, that would take place both in the real world and a digitally created one. Similar to Sword Art Online and the Oasis, using an advanced headset that would have to be developed concurrently, a user would be able to play a video game using their physical body as a controller. Using augmented reality, the game is played in the real world, with digital objects placed in, similar to Pokémon Go. Rather than using created models as characters in the game, it would feature a mix of precreated and scripted characters, and hired actors. Whilst enemies would be created mostly by the software, Non Player Characters would be hired actors with computerized enhancements. This would entail that the game must be played in a monitored small and controlled environment, allowing the easier management of safety.

The issue with Sword Art Online & the Oasis is that their entire worlds are digitally created. This results in heavy computational power needed by the headset to create such lush and fictional worlds. Furthermore, this means players must be tethered to a single area, as there entire field of view is obscured. To fix this, in my opinion, this theoretical concept would use Augmented reality. It would take place mostly in the real world, with game objects added in to simulate the fantastical elements of fiction. While this does mean the worlds are overall less fantasy derived, this would allow players to actually move around and explore the world themselves in order to experience the game. Many people I know, myself included long to explore the world, to be heroes, to be special, to have purpose in their life. While the games of today alleviate this desire, one feels detached from this fantasy due to the physical boundary between one’s eyes and their computer screen. With a virtual reality headset, this physical boundary is removed. With an augmented reality headset, one can feel truly immersed in a world.

But how would this work in actuality? How far away are we from this childhood dream for humanity? The answer: Not as far as many have thought.

The first concept would be the interaction between the game and the real world. While this concept may seem like science fiction to some, it is definitely possible. In fact, in my last Hacakthon I was able to parse a user’s hand gestures and do appropriate actions. With an advanced form of this, we could create a system to use object recognition software currently existing, and combine that with object tracking software to track the user’s body and location in the world, allowing for interactions with a user’s hands, and accurately moving the virtual world in relation to the real world. We see such concepts in games like Pokémon Go, and even in DARgon ThAR DestroyAR, in which my group created such software to keep the game world in place as a player moves around. Such an idea is most definitely possible, yet definitely complex and arduous.

As the game would theoretically use real actors, their interactions happen verbally. Speech recognition and synthesization is definitely an existing concept, with products such as Siri or Alexa, and with the classic fake scripts such as this:

While being created by an artificial intelligence, by studying the speech patterns of President Trump, these software engineers were able to synthesize his voice and speech patterns!

Hopefully, as this specific field advances, we can create realistic tones and inflictions to further make something more realistic. But regardless, with such software, this project can both recognize speech and respond accordingly, and create voices for its inhabitants without the use of many voice actors.

While these features may be possible today with our current technology, in order to be successful, the project would have to be able to interact, not only between the game and the real world, but between each system. This, along with the heavy amount of code needed, would be the main technical hurdle of the program.

However, the main limitation for this project , and for most VR and AR projects, is the hardware. Most virtual reality headsets of today require heavy computational power, resulting in them being constantly tethered to a PC. Furthermore, while they require much computational power, most headsets are still clunky and cumbersome, and can cause nausea in a heavy amount of people, with only one out of ten people experiencing a pleasant time when using the HTV Vive. Such limitations mean that even though such headsets are incredibly expensive, but they also can cause motion sickness in a vast amount of people. As such, the largest hurdle in this potential program comes nothing from the technical aspect, but rather the physical aspect.

But regardless, I’ve spent some time researching the concepts for this idea, and starting the frameworks for object recognition, using my phone in place of a VR or AR headset. By the end of my life, I will work hard enough to fulfill parts of this project that has captivated my interest ever since I could imagine.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.