16 Sep 2023  |   04:17am IST

Finding the right story in mixed reality

Goan author Clyde DeSouza has written three books, ‘Think in 3D,’ ‘Memories with Maya’ and the recently released ‘Storytelling in Mixed Reality’. Interestingly, the same book is also written in Portuguese to contribute to furthering the Goa-Portugal Lusophone ties and rich heritage of storytelling prevalent in both societies

Dolcy D’Cruz

 

 

“W

e’re all storytellers and finally there’s a stage and canvas unlike any other, to allow us to tell the most immersive stories we can think of, via the medium of VR and Mixed Reality,” says Clyde DeSouza, the author of the book, ‘Storytelling in Mixed Reality’.

Originally from Siolim, Clyde was born in Mumbai and brought up in India and Kuwait. However, his work and career as a writer, and immersive media consultant makes him spend time between Dubai, Mumbai and sometimes in SE Asia. ‘Storytelling in Mixed Reality’ is an anthology of topics on immersive media that Clyde has been writing about between 2015 till date. However, to select relevant topics they’d appeal to a wider audience of storytellers it took him about eight months to compile into the final book. “I’ve learnt a lot while researching as it was more hands-on research in the area of how you can “engineer human emotions” via the medium of VR. This is a powerful tool indeed and goes into fascinating territory from PTSD (post-traumatic stress disorder) treatment, to human psychology and education,” says Clyde about the book.

With constant research and closely following up on the latest developments, how does Clyde define mixed reality? “Mixed Reality (MR or Extended Reality; XR) are just fancy terms for Augmented Reality (AR). Now, there is a more technical way of describing mixed reality, and that to me would mean a third person ‘view’ of someone inside of, and interacting with elements in a virtual world. Imagine you’re wearing a Virtual Reality headset which has cameras, such as Meta’s (Facebook’s) Oculus Quest. Now, if while wearing the headset you are in a computer generated ‘room’, then you’re in Virtual Reality. Next, you knock of the walls and the ceiling of the room, thus allowing a view of the real world via the cameras; you’re now experiencing augmented reality, or, Mixed Reality.

‘Storytelling in Mixed Reality’ is also written in Portuguese. He shares why he choose to write in Portuguese. “In ‘Think in 3D’, there’s a chapter on how movies in 3D can help in documenting historic architecture stereoscopically in scenes. One chapter even contains 3D pictures of Old Portuguese architecture in Goa. I wanted this book, ‘Narrativa em Realidade Mista’, to be an original contribution from a Goan author to a language that shares a history with Goa. The preference to translate the book in Portuguese is equally in remembrance of my late father, Gregorio Avelino Desouza, who studied in then Portuguese Goa and spoke fondly of Lisbon. The Portuguese have always been pioneers and navigators, and the art of cinematic Mixed Reality storytelling is so new the world over, why not offer food for thought in the language of those with a pioneering and storytelling spirit. After all, not everyone thinks in English,” he says.

His previous book, ‘Think in 3D’ has been used as a part of the syllabus in the US. However in Goa, schools, higher secondary schools and even colleges don’t have specific courses on VR programs. “Access to this technology should be made available and affordable. With that will come the urge to experiment and invent/innovate. I’m appalled at the kind of customs/ import duties applied to gear that should have exemptions as part of educational resources. The state of Goa can set a precedent and example by looking into this. I was surprised at the price jump in books from Amazon if choosing to have them shipped from the US. A ‘tax’ on knowledge is a new first for me. In the West, ‘Think in 3D’ is being used for the past five years as part of syllabus at Immersive design courses at the University of Southern California, there’s also interest in ‘Storytelling in Mixed Reality’ for an upcoming course in VR production,” explains Clyde.

He further adds, “Think in 3D’ is about stereoscopic 3D filmmaking and the book was aimed at 3D filmmakers. It was used by some of Hollywood’s top directors of 3D. ‘Think in 3D’ is actually complementary to this book as it builds on the immersiveness of storytelling going beyond the cinema screen, truly breaking the 4th wall as it were, and pulling the audiences into the story, storytelling to storyliving.”

Clyde believes that a wide range of readers will find ‘Storytelling in Mixed Reality’ interesting including directors, storytellers, film students and even those in the fields of psychology and medical fields including therapy, behavioural studies. “Basically, anyone looking at immersive storytelling might hopefully benefit from the book. It’s written from a ‘what if’ perspective to seed ideas for furthering VR, AR and Mixed Reality,” he adds.

Since VR, AR and the Metaverse are evolving fields, Clyde confirms that the book will need updating for future editions. “I already know one area that’s ripe for a future edition; ‘Generative AI in Storytelling: From script to immersion’. It’s all greenfield (to borrow an engineering term) and everyone is learning. There are no gurus yet when it comes to VR, AR and the Metaverse. The language and grammar of storytelling in mixed reality is still being written,” concluded Clyde. The book is available on Amazon in Kindle e-book format as well as in Paperback format.