To watch 3D models in VR you need a VR Headset. A thettered headset like a HTC vive or HP Reverb G2 are direct connected to the pc with Revit installed. use Enscape or Vred to watch in VR. These headsets are more expensive thean a Oculus Quest2 standalone headset.
Using a Quest it’s posible to connect your PC in the same network segment over WiFi. You are totaly free to move and can sit everywhere. Connect a bluetooth keyboard and mouse to the headset. The Quest can connect with VRdesktop. On the PC runs the streamer application from VRdesktop. Choose an envirionment in VRdesktop and you see the real desktop from the physical PC. Even the mouse is working!
An drawing opend in Revit is also opend in Enscape as a plugin for Revit. Enscape makes a rendering in VR. Choose the headset and SteamVR is opening.
The file in Revit is a demo from HvA (Hogeschool van Amsterdam) thanks Richard de Nier, BIM consultant.
Revit 2022 and Enscape 3.0 are 14 day trial versions.
Normal video recording in Oculus Quest is a square format. If you want to produce 16:9 format videos, run several scripts in SideQuest. (link: https://sidequestvr.com/ )
Install SideQuest on your PC and set Oculus Quest in Developer mode. After connecting your Quest to the PC, run an custom ADB command:
Je kunt simracen op een monitor, maar die is vaak te klein. Wat als je achter het forcefeedback stuur zit en voor je neus een mega gekromt scherm? Dat kan met een VR bril op.
Met de Oculus Quest2 op WiFi in hetzelfde netwerk als de PC (ik heb een dubbele netwerkkaart die ik om beurten disable of enable) kan een verbinding worden gemaakt met de App VRdesktop naar de PC. De achtergrond op zwart en een zo groot mogelijk desktop scherm. Een beetje gekromt. Dit scherm zet je recht voor je en zo dicht mogelijk bij. Je kunt nu vrij bewegen met je hoofd en toch lijkt het of je in de cockpit zit. Ik had geen last van motion sickness.
Zie deze opname Direct in de Quest2. (instellingen via sideQuest op wide gezet, anders krijg je standaard een vierkante opname.)
2. Kan het ook in VR?
Ik heb een HP Reverb G2 met hoge resolutie, die direct aan de PC is gekoppeld. Als je de applicatie vorpX installeerd, dan kan deze van gewone 2D games een 3D VR game maken. Een ander klein programmaatje (OpenTrack) zorgt ervoor, dat je je hoofd ook een beetje heen en weer kunt bewegen (je kunt dus om de halo heenkijken). Als je TrackIr aanzet in de game (camera settings) werkt het nu ook zonder OpenTrack! Terwijl op het computerscherm het gewone beeld is te zien, is in de VR headset de game in VR. Je zit echt in de cockpit en kan om je heen kijken! Echter afhankelijk van de instellingen kun je wel last krijgen van motion sickness. De refreshrate en FPS moeten zo hoog mogelijk staan. Mijn GTX 1660i is niet de beste kaart, dus dit is net/net niet genoeg. Als de bewegingen niet goed meegaan, moeten je hersenen te veel compenseren, waardoor een misselijkheid op de loer ligt.
Met de Quest kan het ook. De Quest moet dan met een USB3 kabel aan de PC hangen. (in de laatste versie v29 kan dat ook wireless met de WirelessLink setting aan.
Mijn beste ervaring is de eerste configuratie tot nu toe.
Als je met VR bezig bent en dan vooral geïnteresseerd in handtracking op de Oculus Quest, wordt het tijd om eens diep in de materie te duiken. Hoe maak ik zelf een VR app?
In 2015 heb ik al eens een VR/AR masterclass gedaan om meer te weten te komen hoe je een VR/AR experience moet aanpakken. Storytelling en hoe krijg je iemand mee in wat je laat ervaren. Bijvoorbeeld met geluid de aandacht trekken of de rendering beperken met een klein lichtje bijv. van een kaars. Alleen het verlichte deel is dan gerendered en dat scheelt performance, maar ook de focus.
Sinds mijn hernieuwde contact met de sint Maartens Kliniek in Nijmegen, ben ik een handtracking demo app gaan maken voor de Quest. In 2018 heb ik meegewerkt aan een VR4REHAB hackaton bij de sint Maartens Kliniek om VR in te zetten voor revalidatie. De vraag was toen aan het team waar ik zat: Is het mogelijk een VR app te ontwikkelen waar kinderen uit de piratengroep thuis met vriendjes kunnen blijven oefenen. Een goeie continuering van de revalidatie verhoogd enorm de kans om later een normaal leven met de disfunctionerende hand en arm te hebben. Door een hersenbeschadiging (cerebrale parese) is de hand / arm aangedaan. De kinderen oefenen in een piraten pak en revalideren in spelvorm.
In december 2020 heb ik een mogelijke methode gedemonstreerd om op een veilige manier remote te kunnen werken in VR. Daarnaast ben ik begonnen met studeren op het bouwen van een revalidatie spel voor de Oculus Quest. Het doel was handtracking te gebruiken omdat de controllers vaak te groot zijn voor de kinderen. Daarnaast hebben ze ook last van de aangedane hand. Het bedienen van de knoppen is dan lastig.
Na ca. 8 weken studie en bouwen/testen, is het gelukt om een app te bouwen, waar met handtracking ballen, kanonskogels en met blokken kan worden gewerkt. Het wisselen van de scenes gebeurd met pushbuttons en kan zonder het gebruik van controllers worden gedaan. Het heeft mij een hoop inzicht gegeven wat nodig is om je idee om te zetten in een VR app. Handtracking staat nog in de kinderschoenen, maar heeft een grote potentie. Het maakt het werken in VR een stuk natuurlijker.
De eerste versie wacht nu op publishing van Oculus App Lab om op een eenvoudige manier te installeren op de Quest. De demo app is gratis te installeren wanneer de validatie voltooid is:
De app is online in de App Lab store! Dankzij een aanpassing volgens de leasons learned van Tony heb ik de rechten op de mircofoon aangepast in de androidmanifest file. Export de build naar een map en open deze met androidstudio. Wijzig de androidmanifest (delete de rechten op microfoon) en Rebuild de apk. (ook de juiste vertificaat store instellen!)
Blockbusters are made to be seen on the big screen in the cinema, razor sharp and with Dolby Atmos sound. Christopher Nolan is one of the last directors to have had a major cinema premiere with Tenet. He is horrified that his film would be viewed on a telephone on the train. We didn’t campaign for that at beautiful locations for months.
Now that we are glued to home by Covid-19, is VR a possibility to experience the cinema at home? I made an attempt to find out with a VR headset and wireless headphones I already had. New and Legacy gear from the man cave on the attic!
>10 year old wireless Philips headphone with surround soud
What is the setup?
Oculus Quest 2 (light and sharp display of almost 3K per eye).
Game pc with NVidia GTX 1660Ti video card, 32 GB memory and AMD FX6300 6 core CPU, Samsung SSD 860 EVO.
Philips wireless headphones SBC HC8855 with surround sound.
Virtual Desktop runs on the Quest, which connects to the streamer on the PC via WiFi. Environment is dark Cinema. PC desktop can be seen on virtual cinema screen.
PowerDVD20 runs on PC to play all kinds of movies. 3D and 4K. Movies can be played from Blu-ray player with disc or as file.
The sound from the PC does not go via the Quest but directly to the Philips base station that controls the headphones.
Dedicated WiFi router for Quest and PC connection. (Netgear Nighthawk 7000)
To keep the latency on the WiFi as low as possible and to achieve an almost network quality, I used a dedicated WiFi router only for the HMD. The PC has a network connection with this Access Point and is in the same network. First I tried the basic WiFi connection I have.
What is the ultimate goal?
You are in a virtual cinema (unfortunately the only one) to watch a (3D) movie with a surround sound on a large screen. Nice or not? And at your home on the couch.
How is the experience? Scenarios:
1. First, via the existing Netgear Orbi Pro, a connection to 5 GHz WiFi. The latency in the Quest via the Virtual Desktop App is: 21 ms Codec HEVC (auto in streamer)
A. The sound goes to the Quest via WiFi. Headphones directly connected to the Quest. Latency is: 21 ms
B. Mute sound to Quest in the Desktop Streamer and the headphones directly on the PC. Latency is: 22 ms with film running 26 ms
2. Connect Quest and PC to a dedicated Access point (Netgear R7000). The latency measured in the Quest Virtual Desktop App is: 18-22 ms with movie 25 ms, Codec on H264 (we now have a dedicated network). Connection speed WiFi is 860 Mbps
A. Let sound run directly through the PC, so that the stream only contains images.
• DFS (Dynamic Frequency Selection channels on 80 MHz (Quest uses 80 MHz) set 5 GHz to one of these channels 58, 106, 122 to use. (Channel 56 used. (no DFS channel 58 available)
Result and best experience:
Both options with WiFi had a good performance!
Option 1 B:
Sending sound to wireless headphones via the PC and switching it off in the VD streamer is the best audio experience. Image in Quest at 90 Hz.
The Quest only gets image. Connecting the headphones to the Quest did not produce good sound. Wired headphones on the Quest are possible, but no surround sound.
Unfortunately, the wireless headphones cannot be connected to the Quest. Because of this you are condemned to sit near the PC anyway. With the Orbi Pro router and satellite, an average bit rate of 650 Mbps can be achieved at 5 GHz. The 90 Hz option in the VD app did not seem to work, although Quest and the VD were set to 90 Hz.
PowerDVD20 does play content in the Quest when a movie is stored as a file on the PC, but when a Blu-Ray is started in a player, it can only be seen on the screen. It cannot be seen within the headset. Also tried with another stream program like vSpatial, but no active Blu-Ray screen here either. It seems that the streaming software does not transmit a Blu-ray on this screen. It can also be restricted within PowerDVD20. DVD does work.
Option 2 A:
Even with AP and headphone transmitter both on the attic, I could sit downstairs (two concrete floors) in a relax chair watching movies with great video and audio quality.
If it’s
possible to work everywhere with your personal remote office in a Headset and
the possibility to work together with colleges by sharing screens and
videoconferencing, then he has a point.
Nowadays
people works in smaller teams by scrum squads and mostly remote to datacenters.
So you can work remote from other locations and also connect colleges any
moment.
What is mostly in place now?
Remote workers can use a VPN connection to connect their business applications and mail, login to remote virtual desktops, Apps by VMware WorkspaceOne or connect by Citrix to business Apps and more. Communication by (mobile) phone and skype or facetime or other videoconferencing tools.
Most work
is output driven and can be don also on every remote workspace. You only need a
good laptop or tablet with internet connection. Most people do maybe already of
their work on a smartphone. With the intelligence of the smartphone it is
sometimes faster to write an article than on a laptop.
New remote workspaces.
With the
introduction of VR and AR, we can connect in the headset to a remote workspace.
Like the home office, we can connect to a virtual desktop in a datacenter
(Company office) and do our work. With the ongoing development of the headsets,
the quality and easy to where is no problem at all to work for longer periods
in a VR world. In your virtual office (looks like you are anywhere in the
world) you have the company work environment in front of you. Collaboration
with peers is just a click away.
Now we have
simple avatars, but in the future this projection of you is real-time and in
real person. Face expressions will be available.
People from
around the world can join teams from everywhere. Working on 3D objects in VR is
possible, not only working on documents! The housing problem Mark is thinking
about could also a small home in India.
With AR we can join teams in real life with colleges in the same room. Work on the same project that is projected in our space. Matter of time to use this. Designing cars and step into the just build interior is one of the benefits.
See Varjo predictions Varjo’s headset has an extra display in the centre of your eye with retina resolution, so ultra sharp vision.
How long does it takes? Some people say 5 years other 10 years. All great changes and inventions takes time. Looking back it’s easy to see what the difficulties are and how long it takes to embraces new methods. Mark focus is 2030.
My first experience with Quest Link is just amazing. After update to firmware 11.0.0.180 in the Quest and installed the Oculus Home software on my PC, should Link worked? I had ordered an USB 3.0 extension cable and a USB 3.1 A to C connector. Total length is 5+1 = 6 meter.
PC configuration is not the latest, so would it work? ASUS
mainboard M5A97 EVO R2.0 has two USB 3.0 ports. The free one has a connection,
but when started in Quest, the application on the PC is moving, but the screen
in the Quest was black. I checked the log files in Oculus application and found
that my ASMedia USB controller is supported. Removed the WiFi USB dongle and
replaced it with the Quest cable, now Link was working!
With the Velcro cable binder
its easy to fixed the cable to the headset.
Only the warning PC is not compatible with the minimal requirements is a small problem, but did not harm the experience. The AMD FX-6300 Processor is not compatible. Maybe hack the configuration json file in Oculus Software.
note: add processor type AMD FX-6300 did not work in config file.
My Configuration:
PC specs: ASUS M5A97 EVO R2.0 , AMD FX-6300 6 core 3.51 GHz , 32 GB mem, NVIDIA GeForce GTX 1660 Ti 6
GB mem.
Quest connection to PC: 5 meter DeLock Extension cable USB
3.0 A to USB 3.0 A female, USB 3.1 A to USB 3.1 C (Allekabels.nl) Note: charge cable that comes with Quest is
not usable.
Test applications in
Oculus on PC:
On Quest I had Apollo 11 Quest running, so the PC version Apollo11 VR HD must be compared. This VR experience is much more detailed then the Quest version. Amazing how good the OLED display of the Quest is. Now the PC is processing the game in a detailed image.
The same with Anne Frank House, much more details.
FlyInside Flight Simulator works with the virtual Oculus controllers, but no hands!
vSpatial virtual desktop in beta on the Quest I tested over WiFi and now over the link with the Oculus version. Both apps connect to a virtual Windows10 64 bit Desktop running on VMware ESXi6.7 in my homelab. vSpatial Remote Manger is v1.0.7255.4019 is installed in the VM. Runs with no latency at all.
Nice to play with Aircar. Sometimes Quest is freezing, while PC runs. Not available on Quest.
Blade Runner is also a nice game to play on PC. Not available on Quest
vSpatial screen capture with CyberLink, just the running application on PC. Left screen is the physical Windows10 PC and the right screen is the virtual Windows10 desktop on VMware in my homelab. ( In Quest Bluetooth mouse and keyboard not working.)
Running Virtual Desktop in the Quest and added streaming apk with SideQuest, SteamVR games are running in the Quest. Screen Quest is live recorded in Quest itself !
I played some VR games like FlyInside and Google Earth VR in the Quest and was surprised by the quality. FlyInside is an flight sim where you really sitting in the cockpit. Most of buttons and handles are activated by touching with your virtual hands. Even sitting on a fixed chair, it feels very immersive to dive in a turn.
Flying in Google Earth VR is amazing. Search your home and fly around in 3D. Step in the 360 streetview and be really there.
The larger the monitor (or more), the more efficient you can work, has been known for years. Multiple application Windows side by side works a lot faster. What if you have a super large screen and a high resolution? What does that extra convenience cost? Is that possible for € 500?
Since a month I have had the Oculus Quest, the cable-less VR
headset from Oculus. This 6dof HMD (6 degree of freedom head mounted display)
with OLED screen is extremely suitable for use with a virtual desktop. The
headset is connected via 5 GHz Wi-Fi network to a PC in the same network. The
Quest App Virtual Desktop connects to the PC on which a stream client is
installed. You can connect up to 4 PCs in this way. The video card of the PC sends
a high resolution image to the headset. So you have a super monitor on your PC
but then in Virtual Reality!
Everything that runs on your PC can now also be seen in the Oculus Quest. From all games to standard programs such as Word and the browser. Even 3D side by side movies are played in 3D with a VLC player. The environment in which that can be seen, can be an Auditorium or cinema. You sitting in front of the large screen, but also a virtual office is possible.
Working on the VR
desktop
How can you type? Typing texts is done with the controllers,
which takes a virtual keyboard and allows you to type two fingers. If you can
type blind, then a Bluetooth keyboard can come true. Furthermore, with the
pointer of the controller you can simply open all programs as if you were using
a mouse.
You will think nicely, work in my own bubble, but what will it cost? First you need an Oculus Quest for € 450 (64 GB mem, you stream everything) and Virtual Desktop App of € 19.99 via the Oculus store. You can download the stream agents for different OSs on the Virtual Desktop site. You can also download the apk that Oculus had ordered to remove from the App: SteamVR games can also be streamed in VR to the headset! This apk can be placed on your HMD with SideQuest.
Once you have started the Virtual Desktop App you can choose
which PC you want to connect, but that can also become automatic if you only
have 1 PC of course. The image quality can be adjusted, but depends on the
video card. I have a GeForce GTX 1660 Ti, the new NVidia without raytracing, comparable
to the GTX 1070 Ti. You can see in the Virtual Desktop settings screen that the
GPU is doing a good job. Ca. 80% with a frame rate of 72 Hz.
VMware Horizon 7.9 virtual desktops
In addition to a physical PC connection, I have also connected a virtual desktop of VMware Horizon 7.8 with two monitors. Just add a streaming agent in the Linked Clone and AppVolumes in the Appstack with agents and ensure that the desktop is in the same network as the Wi-Fi of the Headset. Connect in the headset and you will see one monitor, but can switch between the available monitors. (or 4 in Horizon 7.9) screens. On a mobile headset like the Oculus Quest is only one screen visible because the processor power of the headset is less than an HMD like Oculus Rift, that is tethered to an GPU in the PC. Quality of course dependents on the video card in the server. A vGPU system is highly recommended. VMware Horizon 7.9 supports now NVidia T4 cards.
note: only NVidia drivers with vGPU pass-trough can accessed. VMware VGA driver cannot connect!
My last presentation on the VMUG TechCon in March was all about how the journey of VR/AR is running into the future. After a few years there are several new products to make 2019 the year of VR.
VR creates an immersive virtual world. AR let you enhance the real world with digital add-ons. Better display’s, wireless and easy to use 6dof (6 degree of freedom) are available now. The best HMD (head mounted displays) are the HMD with more than 180 degree FOV (field of view). StarVR, X-TAL and consumer friendly PIMAX produce these. All connected to a big PC.
I was waiting for the new Oculus Quest and happy to be an owner now. After a few weeks working with this device, I’m very excited about the potential of this headset. No wires, easy setup and 6dof. Streaming the user view to Chromecast v3 on a big TV is great for the others, watching why you are running around with funny moves.
Everything is bigger and immersed in the headset: You don’t
need a big TV (3D) to watch movies ore Netflix. 360 photos and videos are now
the ‘real’ world. With a stream application ALVR, it’s possible to stream
content from you PC. SteamVR games run in the Quest. Not all high end games,
but Google Earth VR is working well. Fly over your hometown in 3D and visit places
on the whole world. Even the Desktop of your PC is on a very big screen
available.
What if we can create a virtual workspace in VR with all the screens we need (more is more productive), team meetings virtual in the same room we live video communication? It’s possible with ‘BigScreen’ but more professional in vSpatial. This could be the workspace of the future. Running virtual desktops in the cloud and share with the team. vSpatial is working on the Quest version. I can’t wait its ready.
Flying around the world to meet members in the project is
reduced with this technology. It save more time and CO2 , save the
environment. Just to drink a beer together we must physically meet, so do not
replace all by virtual.
All my earlier bought stuff and profession experience
is coming together in the Oculus Quest. My first 3D video camera, Samsung Gear360
camera and virtual desktops. If you see the right path, every investment in
time and money is worth of it. Join the journey!
