6 ms·

Logitech announces a development kit for using keyboards in virtual reality

95 points by haydenlee 9y ago

mlevental 9y ago
does this just display a keyboard in the field of view of the vive perspective? why do i need to see my keyboard if i know how to touch type?
- bhhaskin 9y ago
  Touch typing while being pretty much blind is very difficult. It can be hard to even find the keyboard.
  - pmoriarty 9y ago
    I touch-type in the dark without seeing the keyboard just fine. But that's on a physical keyboard that has something which virtual keyboards lack: tactile feedback. Tactile feedback is one of the missing pieces of today's VR devices. Without it typing on a fully virtual keyboard is just not going to be nearly as accurate as on a physical keyboard, and providing visual feedback is just about the only thing you can do about it.
- melling 9y ago
  A few ideas mentioned in the article sound interesting: “But VR can transform and augment that trusty keyboard – so easy to disregard – into a contextually aware companion for whatever application you use, becoming a palette for your creative workflow, dynamically providing you with any commands and shortcuts you need.”
- devdoomari 9y ago
  why even assume you have to 'look at' the keyboard? the keyboard can follow the user's view... along with the silhouette of the user's hand...
  - pmoriarty 9y ago
    This could be done, but I expect it to be usually pretty annoying. Most of the time when I type I don't want to be looking at my hands or the keyboard. Instead, I want to be looking at either the text I'm typing or something else (like some other text, or a video, etc). Having a silhouette of my hands and a keyboard superimposed over what I'm looking at will just be distracting. That said, I don't really have a better suggestion. Typing in VR is just going to be plain painful for anyone who can touch type with any speed on a physical keyboard.
    - arbie 9y ago
      With gaze tracking, you could have hot zones bound to different shortcuts, one of which can be "Toggle floating keyboard".
- shpx 9y ago
  I thought the same thing so I disabled the backlight on my laptop's keyboard. After accidentally turning it on in the dark a month later I will never go back to a non back-lit keyboard. It's hard to place your hands on the keyboard in pitch black or when you're in a weird position like lying on your back with the laptop on your chest. If you've ever partially closed your laptop so that the monitor would shine on your keyboard so you could find 'f' and 'j', you know exactly what I'm talking about. And also TFA says "for a true typing experience you need to see your hands, and we’ve created a way to use the Vive’s existing tracking to do that".
  - pmoriarty 9y ago
    The way I know where to place my fingers on a dark keyboard is by feeling for some raised bumps which are on the f and j keys on my keyboard. As long as I've got those, I can always easily put my fingers in the default position on the home row, and from there it's easy. As long as I can use such markers to position my hands in the proper initial position, I really don't like to have a lit keyboard in the dark. It's distracting and annoying. If your keyboard doesn't have such bumps it's easy enough to make them either using some craft materials you can stick on the keyboard (like some tape even) or by cutting small notches in to it using a file or a dremel tool.
    - deleted 9y ago
      [deleted]
    - 83457 9y ago
      My dad had a keyboard with those bumps rubbed almost completely off. Occasionally I would need to use the computer attached to it and found it really hard to use. I'm so accustomed to unconsciously placing my fingers correctly using them that without them I was really ineffective with the computer and found it frustrating.
- klibertp 9y ago
  A lot of people can't touch-type. I can't, not really, I need to glance at the keyboard once in a while. My typing speed is comparable to touch-typists', which is why I never felt the need to learn to properly touch-type. The need of seeing the keyboard is problematic in very few situations in practice, and with a back-lit keyboard it's never a problem... as long as seeing my keyboard is possible at all, which was not the case in VR. I could invest a lot of time to unlearn my current way of typing and learn proper touch-typing technique, but that's a lot of work and doing it just to be able to type in VR feels like a waste of time. With this tech I'd be able to work in VR without changing my way of typing. To me, this makes using VR for work practical for the first time. It's actually huge, and if it works well it's, to me and others in similar situation, potentially life-changing tech.
  - deleted 9y ago
    [deleted]
- tudorw 9y ago
  strictly off-topic, but has anyone else noticed on the Yoga Book (the 'Halo' Keyboard') that they screen printed the physical locator nibs from the F and J onto the flat keyboard :)
- lps41 9y ago
  Gaming with peripherals. If you are playing a VR game where you are using a joystick or some other peripheral, you still may need to use your keyboard for some actions, in-game chat, or simply to pull up a browser and google something while in-game. If you primarily have your hands on a joystick or game controller, you would have to take your VR headset off just to figure out where your keyboard/mouse are.
giraffee 9y ago
Seems difficult to type only by touch in dark.
- microcolonel 9y ago
  I honestly don't understand how this is difficult for people. I can touch type without looking, on a keyboard which is moving randomly. It's not a parlour trick, it's just what's supposed to happen if you type much at all. If you can't touch type in the dark, there's something wrong with your skill or your equipment.
  - pmontra 9y ago
    I can touch type but not everybody can. Look at non sw developers using computers: usually they look for keys and are slow. They are the huge majority of the market. I'll be like them if I were not using a keyboard all the time. Btw, this is a VR keyboard from the Dennou Coil anime 2009. Nobody was using real reality (pun intended) laptops there, only VR ones. I guess they were running on some AWS like cloud. https://giphy.com/gifs/typing-keyboard-dennou-coil-1337mjZhdNJWSY https://giphy.com/gifs/typing-keyboard-dennou-coil-1337mjZhd...
    - hobofan 9y ago
      > Look at non sw developers using computers: usually they look for keys and are slow. They are the huge majority of the market. Citation needed. (Seriously, I was looking for statistics on that recently and couldn't find any)
- deleted 9y ago
  [deleted]
- lancebeet 9y ago
  I think this issue is caused partly by the design of modern keyboards. Back in the day, keyboards would have clear, lasting, physical markings on the F and J keys and the 5 key on the numpad. They also had special keys that were easily recognized for caps lock, return and so on. This meant you could put your hands almost anywhere on the keyboard and always know where you were. Finding the home row without looking took a fraction of a second. Another feature that keyboards used to have, which is becoming increasingly uncommon today, is grouping of the function keys into fours. This meant you could always press the right function key without looking. Modern keyboards more often than not don't do this, and reliably pressing the right function key without looking is nearly impossible today. Old keyboards were (generally) designed to allow for touch-typing. Modern keyboards are (generally) designed to be looked at.
swalsh 9y ago
Combine this with the new Pimax, and infinite workspaces start to become a practical reality. Which could help propgate the technology more which would lead seamlessly into more teleconferences, which have a feeling of personal interactions... which is important, even if we as programmers don't always appreciate it. The other day I was playing a game of onward. It was the rare instance that a team wanted to work together on a shared goal. In the 15 seconds before the match, while planning some tactics... it occured to me how natural the meeting felt. It was like we were all in the same room. I felt engaged with EVERYONE. That doesn't happen on conference calls. I work from home full time, I think the biggest downside are the missing meaningful interactions with other people. I think VR has huge potential to bridge that gap.
- kirrent 9y ago
  I love that natural feeling of interaction with other people in VR. When playing in Rec Room so often you take a break from games to just have a chat with some other people because the interaction is somehow so engaging despite the cartoony graphics.
  - Rayz0r41 9y ago
    Same here - I usually avoid multiplayer games altogether but this weekend I discovered VR poker and was utterly immersed - am playing in my first tournament tonight!
- pmoriarty 9y ago
  "I work from home full time, I think the biggest downside are the missing meaningful interactions with other people. I think VR has huge potential to bridge that gap." VR also has the potential to be way more addictive than current forms of online interaction and entertainment. I'm reminded of I think it was a Larry Niven story (maybe Ringworld) in which the protagonist has an electrical wire put in to the pleasure center of his brain and he just sits home and pushes the button that activates it for weeks on end. Technology is getting closer and closer to that.
  - 9y ago
joshumax 9y ago
An I the only one who lost the ability to touch type for a moment after reading this article?
wwwigham 9y ago
I'm more impressed by the videos of the hand tracking around the keyboard than the tracker in the keyboard! Is that fidelity just using the Vive's outward facing camera? They don't show any tracking gloves in the mockups.
- haydenlee 9y ago
  Yes it's using the Vive camera
- jayd16 9y ago
  Its actually pretty simple. If you know where the keyboard is, you can clip out that part of the camera's input. Then you just need some edge detection to get the hand silhouettes. And actually...that seems like a portable idea. So I suppose you could do this for any tracked thing.
jlebrech 9y ago
now we can properly have ides and desktops in VR. but i'd would rather use a wider range of motion (at some point, if someone makes that) to input to stay active.
veli_joza 9y ago
Just look at this nonsense [1]. We have brand new technology with unlimited potential for new abstractions and paradigms. So, what do we do? We make a virtual desktop workstation, the same one we had since 70s! We limit all rendering to a 2D surface, make it curved and put a nebula in the background. Is this really the cutting edge of information technologies? Also, we need both mechanical keyboard to sense key presses and 3D tracking of finger movements? I get it, Logitech wants to keep selling keyboards, but for VR experience I would rather have tracking of facial features and eye movement. [1] https://d201n44z4ifond.cloudfront.net/wp-content/uploads/sites/6/2017/11/02095838/Logitech_G_Bridge_VR_Keyboard_Hands_A.gif https://d201n44z4ifond.cloudfront.net/wp-content/uploads/sit...
- csmattryder 9y ago
  Exactly! One of the first things I did with my Rift (well, after Superhot...) was try out the Virtual Desktop kit. It's lacking. I thought it'd give me a 3D frustrum to throw window back/forward, drop Spotify out to the peripheral, have VisStudio in glorious megapixel size. Instead, you get your regular desktop screens mirrored and constrained to a 2D plane. Gods help you if you've got mismatched DPI screens, like my 1080p & 2160p pair. (Go on, I'll await some smart sod to tell me "Product" is exactly what I'm looking for...)
- ewjordan 9y ago
  I absolutely do not want facial features and eye movement to completely replace finger movement for input. I can neither control my eyes nor face at anywhere near the rate that I can control my fingers. If VR wants to replace my desktop environment for anything other than games, it'll need to retain some sort of high fidelity, high frequency input vector.
  - veli_joza 9y ago
    Yeah, I wouldn't use facial and eye tracking instead of traditional input interface. They are meant to enrich interaction with other people. Eye tracking could be used for some optimizations, like dynamic LoD based on user focus. I thought about input interface that could replace keyboard for VR and mobile devices. Brain implant would be perfect, but that's not feasible with today's technology. Voice is good contender, except for privacy issues, and it's not really usable in work environments. Keyboards are our best solution so far, but they don't evolve and to me they seem like a dead end (plus, they are not portable and require both hands to use effectively). I was thinking about touch sensitive surface that recognizes drawn glyphs. The idea is that anyone who is literate can start using it without any training, just draw letters instead of typing them. With machine learning and some clever visual feedback, both user and machine could adapt to each other to increase the input speed (by simplifying glyphs and by defining 'snippets'). Interface could be expanded to both hands to double the speed, but it would be completely functional with just one hand. It could even be used blindly. For mobile devices, such sensitive surface could be placed on backside - virtual keyboards that take half of screen are just horrible.
return0 9y ago
I don't think non-fanatics are willing to go through such painful process for a large field of view. Maybe it would be more profitable to start working on a 360 projection screen.
loup-vaillant 9y ago
Wait a minute, fine grained hand tracking?
BatFastard 9y ago
As someone who has been a developer of virtual worlds for 17 years. I can say that this is a very incremental change, as all of the changes have been since 1994. Personally I think we need to reimagine interfaces to the world around us, even the virtual worlds (VR/AR/MR) around us. Voice input, AI, hand sensing technologies could make for new ways for changing the worlds. The book Daemon by Danial Suarez and his eSpace holds much promise. I have been experimenting with those idea in a new VR world I have been working on for a few years.
- jayd16 9y ago
  A lot of these technologies are being developed and its pretty awesome. However, I'll argue there's clearly space for a keyboard in the virtual world. If you want to sculpt a model in the real world, you'd use clay and your hands. We can use VR to move away from KBM interaction for things like that. If you want to write a book in the real world you...type it out. Its the best way we know to get text out of your brain and somewhere else. Don't throw the baby out with the bathwater.
  - BatFastard 9y ago
    Indeed everything in its place. However I really don't think I would go into a virtual world to write a book, or do any other kind of activity that requires a lot of keyboard interaction. I used to hope for the unlimited monitor space, now I am not so sure how practical that really is.
- gervase 9y ago
  I think motion sensing is probably the way forward; voice control is intrinsically limiting in most (crowded) environments where computers are used today. Considering humans can ride bikes, drive cars, play instruments, etc. (including typing on keyboards!), I think that indicates that non-verbal, physical interaction is not nearly saturated as a transmission channel. Conversely, it's hard to imagine someone verbalizing "navigate to HN" in a loud open-space office, or "Excel, create a pivot table" or whatever. I think it's fine in private spaces like your home, but in public spaces, you're implicitly broadcasting your activity to everyone around you, which I consider to be a strong negative.
jefurii 9y ago
I could see the hand tracking being useful for people who don't know how to touch type (and have to look at their keyboard and fingers) and who need to type while using a headset.
DoctorMemory 9y ago
I understand Logitech wants to keep selling keyboards and I agree that keyboards are currently the fastest way to enter text. However, it IS the wrong direction. The better direction is what will be opened with tools such as the Vive Knuckles enabling a more complete language of gestures. It seems to me that the better way forward is to stop clinging to an old paradigm that requires bulky equipment that can obstruct the user's volumetric interactions and instead to build off of those volumetric interactions even though there is a cost of a learning curve. I doubt businesses will implement it since they run the risk of alienating current gen customers and thus losing money. I think there will be some VR experiences aimed at younger customers that will implement it and over the next decade we will wonder why we ever tried to bring a keyboard into VR/AR instead of just using our hands.
mncharity 9y ago
If you want to explore this kind of thing, you can mount a camera on your HMD (Vive's is crippled), use a WebVR stack (simple), track objects using visual markers and javascript tracking libraries (jsartoolkit5 and/or tracking.js), and do selective camera pass-through AR. It's crufty, but not hard. EDIT: You can simply use the Vive's camera, with tracking.js color tracking, especially with a small minDimension (number of pixels) threshold. Yellow is good. https://artoolkit.github.io/jsartoolkit5/examples/ https://artoolkit.github.io/jsartoolkit5/examples/ https://trackingjs.com/examples/color_camera.html https://trackingjs.com/examples/color_camera.html ; http://www.keyboard-layout-editor.com/#/ http://www.keyboard-layout-editor.com/#/ https://joric.org/keycaps/#GB-Retro-DSA https://joric.org/keycaps/#GB-Retro-DSA etc
ilaksh 9y ago
How about Logitech makes an affordable effective precise VR glove. Then we can use that for many things besides just data entry.

95 points by haydenlee 9y ago

does this just display a keyboard in the field of view of the vive perspective? why do i need to see my keyboard if i know how to touch type?

bhhaskin 9y ago
Touch typing while being pretty much blind is very difficult. It can be hard to even find the keyboard.
- pmoriarty 9y ago
  I touch-type in the dark without seeing the keyboard just fine. But that's on a physical keyboard that has something which virtual keyboards lack: tactile feedback. Tactile feedback is one of the missing pieces of today's VR devices. Without it typing on a fully virtual keyboard is just not going to be nearly as accurate as on a physical keyboard, and providing visual feedback is just about the only thing you can do about it.
melling 9y ago
A few ideas mentioned in the article sound interesting: “But VR can transform and augment that trusty keyboard – so easy to disregard – into a contextually aware companion for whatever application you use, becoming a palette for your creative workflow, dynamically providing you with any commands and shortcuts you need.”
devdoomari 9y ago
why even assume you have to 'look at' the keyboard? the keyboard can follow the user's view... along with the silhouette of the user's hand...
- pmoriarty 9y ago
  This could be done, but I expect it to be usually pretty annoying. Most of the time when I type I don't want to be looking at my hands or the keyboard. Instead, I want to be looking at either the text I'm typing or something else (like some other text, or a video, etc). Having a silhouette of my hands and a keyboard superimposed over what I'm looking at will just be distracting. That said, I don't really have a better suggestion. Typing in VR is just going to be plain painful for anyone who can touch type with any speed on a physical keyboard.
  - arbie 9y ago
    With gaze tracking, you could have hot zones bound to different shortcuts, one of which can be "Toggle floating keyboard".
shpx 9y ago
I thought the same thing so I disabled the backlight on my laptop's keyboard. After accidentally turning it on in the dark a month later I will never go back to a non back-lit keyboard. It's hard to place your hands on the keyboard in pitch black or when you're in a weird position like lying on your back with the laptop on your chest. If you've ever partially closed your laptop so that the monitor would shine on your keyboard so you could find 'f' and 'j', you know exactly what I'm talking about. And also TFA says "for a true typing experience you need to see your hands, and we’ve created a way to use the Vive’s existing tracking to do that".
- pmoriarty 9y ago
  The way I know where to place my fingers on a dark keyboard is by feeling for some raised bumps which are on the f and j keys on my keyboard. As long as I've got those, I can always easily put my fingers in the default position on the home row, and from there it's easy. As long as I can use such markers to position my hands in the proper initial position, I really don't like to have a lit keyboard in the dark. It's distracting and annoying. If your keyboard doesn't have such bumps it's easy enough to make them either using some craft materials you can stick on the keyboard (like some tape even) or by cutting small notches in to it using a file or a dremel tool.
  - deleted 9y ago
    [deleted]
  - 83457 9y ago
    My dad had a keyboard with those bumps rubbed almost completely off. Occasionally I would need to use the computer attached to it and found it really hard to use. I'm so accustomed to unconsciously placing my fingers correctly using them that without them I was really ineffective with the computer and found it frustrating.
klibertp 9y ago
A lot of people can't touch-type. I can't, not really, I need to glance at the keyboard once in a while. My typing speed is comparable to touch-typists', which is why I never felt the need to learn to properly touch-type. The need of seeing the keyboard is problematic in very few situations in practice, and with a back-lit keyboard it's never a problem... as long as seeing my keyboard is possible at all, which was not the case in VR. I could invest a lot of time to unlearn my current way of typing and learn proper touch-typing technique, but that's a lot of work and doing it just to be able to type in VR feels like a waste of time. With this tech I'd be able to work in VR without changing my way of typing. To me, this makes using VR for work practical for the first time. It's actually huge, and if it works well it's, to me and others in similar situation, potentially life-changing tech.
- deleted 9y ago
  [deleted]
tudorw 9y ago
strictly off-topic, but has anyone else noticed on the Yoga Book (the 'Halo' Keyboard') that they screen printed the physical locator nibs from the F and J onto the flat keyboard :)
lps41 9y ago
Gaming with peripherals. If you are playing a VR game where you are using a joystick or some other peripheral, you still may need to use your keyboard for some actions, in-game chat, or simply to pull up a browser and google something while in-game. If you primarily have your hands on a joystick or game controller, you would have to take your VR headset off just to figure out where your keyboard/mouse are.

giraffee 9y ago

Seems difficult to type only by touch in dark.

microcolonel 9y ago
I honestly don't understand how this is difficult for people. I can touch type without looking, on a keyboard which is moving randomly. It's not a parlour trick, it's just what's supposed to happen if you type much at all. If you can't touch type in the dark, there's something wrong with your skill or your equipment.
- pmontra 9y ago
  I can touch type but not everybody can. Look at non sw developers using computers: usually they look for keys and are slow. They are the huge majority of the market. I'll be like them if I were not using a keyboard all the time. Btw, this is a VR keyboard from the Dennou Coil anime 2009. Nobody was using real reality (pun intended) laptops there, only VR ones. I guess they were running on some AWS like cloud. https://giphy.com/gifs/typing-keyboard-dennou-coil-1337mjZhdNJWSY https://giphy.com/gifs/typing-keyboard-dennou-coil-1337mjZhd...
  - hobofan 9y ago
    > Look at non sw developers using computers: usually they look for keys and are slow. They are the huge majority of the market. Citation needed. (Seriously, I was looking for statistics on that recently and couldn't find any)
deleted 9y ago
[deleted]
lancebeet 9y ago
I think this issue is caused partly by the design of modern keyboards. Back in the day, keyboards would have clear, lasting, physical markings on the F and J keys and the 5 key on the numpad. They also had special keys that were easily recognized for caps lock, return and so on. This meant you could put your hands almost anywhere on the keyboard and always know where you were. Finding the home row without looking took a fraction of a second. Another feature that keyboards used to have, which is becoming increasingly uncommon today, is grouping of the function keys into fours. This meant you could always press the right function key without looking. Modern keyboards more often than not don't do this, and reliably pressing the right function key without looking is nearly impossible today. Old keyboards were (generally) designed to allow for touch-typing. Modern keyboards are (generally) designed to be looked at.

swalsh 9y ago

Combine this with the new Pimax, and infinite workspaces start to become a practical reality. Which could help propgate the technology more which would lead seamlessly into more teleconferences, which have a feeling of personal interactions... which is important, even if we as programmers don't always appreciate it. The other day I was playing a game of onward. It was the rare instance that a team wanted to work together on a shared goal. In the 15 seconds before the match, while planning some tactics... it occured to me how natural the meeting felt. It was like we were all in the same room. I felt engaged with EVERYONE. That doesn't happen on conference calls. I work from home full time, I think the biggest downside are the missing meaningful interactions with other people. I think VR has huge potential to bridge that gap.

kirrent 9y ago
I love that natural feeling of interaction with other people in VR. When playing in Rec Room so often you take a break from games to just have a chat with some other people because the interaction is somehow so engaging despite the cartoony graphics.
- Rayz0r41 9y ago
  Same here - I usually avoid multiplayer games altogether but this weekend I discovered VR poker and was utterly immersed - am playing in my first tournament tonight!
pmoriarty 9y ago
"I work from home full time, I think the biggest downside are the missing meaningful interactions with other people. I think VR has huge potential to bridge that gap." VR also has the potential to be way more addictive than current forms of online interaction and entertainment. I'm reminded of I think it was a Larry Niven story (maybe Ringworld) in which the protagonist has an electrical wire put in to the pleasure center of his brain and he just sits home and pushes the button that activates it for weeks on end. Technology is getting closer and closer to that.
- 9y ago

joshumax 9y ago

An I the only one who lost the ability to touch type for a moment after reading this article?

wwwigham 9y ago

I'm more impressed by the videos of the hand tracking around the keyboard than the tracker in the keyboard! Is that fidelity just using the Vive's outward facing camera? They don't show any tracking gloves in the mockups.

haydenlee 9y ago
Yes it's using the Vive camera
jayd16 9y ago
Its actually pretty simple. If you know where the keyboard is, you can clip out that part of the camera's input. Then you just need some edge detection to get the hand silhouettes. And actually...that seems like a portable idea. So I suppose you could do this for any tracked thing.

jlebrech 9y ago

now we can properly have ides and desktops in VR. but i'd would rather use a wider range of motion (at some point, if someone makes that) to input to stay active.

veli_joza 9y ago

Just look at this nonsense [1]. We have brand new technology with unlimited potential for new abstractions and paradigms. So, what do we do? We make a virtual desktop workstation, the same one we had since 70s! We limit all rendering to a 2D surface, make it curved and put a nebula in the background. Is this really the cutting edge of information technologies? Also, we need both mechanical keyboard to sense key presses and 3D tracking of finger movements? I get it, Logitech wants to keep selling keyboards, but for VR experience I would rather have tracking of facial features and eye movement. [1] https://d201n44z4ifond.cloudfront.net/wp-content/uploads/sites/6/2017/11/02095838/Logitech_G_Bridge_VR_Keyboard_Hands_A.gif https://d201n44z4ifond.cloudfront.net/wp-content/uploads/sit...

csmattryder 9y ago
Exactly! One of the first things I did with my Rift (well, after Superhot...) was try out the Virtual Desktop kit. It's lacking. I thought it'd give me a 3D frustrum to throw window back/forward, drop Spotify out to the peripheral, have VisStudio in glorious megapixel size. Instead, you get your regular desktop screens mirrored and constrained to a 2D plane. Gods help you if you've got mismatched DPI screens, like my 1080p & 2160p pair. (Go on, I'll await some smart sod to tell me "Product" is exactly what I'm looking for...)
ewjordan 9y ago
I absolutely do not want facial features and eye movement to completely replace finger movement for input. I can neither control my eyes nor face at anywhere near the rate that I can control my fingers. If VR wants to replace my desktop environment for anything other than games, it'll need to retain some sort of high fidelity, high frequency input vector.
- veli_joza 9y ago
  Yeah, I wouldn't use facial and eye tracking instead of traditional input interface. They are meant to enrich interaction with other people. Eye tracking could be used for some optimizations, like dynamic LoD based on user focus. I thought about input interface that could replace keyboard for VR and mobile devices. Brain implant would be perfect, but that's not feasible with today's technology. Voice is good contender, except for privacy issues, and it's not really usable in work environments. Keyboards are our best solution so far, but they don't evolve and to me they seem like a dead end (plus, they are not portable and require both hands to use effectively). I was thinking about touch sensitive surface that recognizes drawn glyphs. The idea is that anyone who is literate can start using it without any training, just draw letters instead of typing them. With machine learning and some clever visual feedback, both user and machine could adapt to each other to increase the input speed (by simplifying glyphs and by defining 'snippets'). Interface could be expanded to both hands to double the speed, but it would be completely functional with just one hand. It could even be used blindly. For mobile devices, such sensitive surface could be placed on backside - virtual keyboards that take half of screen are just horrible.

return0 9y ago

I don't think non-fanatics are willing to go through such painful process for a large field of view. Maybe it would be more profitable to start working on a 360 projection screen.

loup-vaillant 9y ago

Wait a minute, fine grained hand tracking?

BatFastard 9y ago

As someone who has been a developer of virtual worlds for 17 years. I can say that this is a very incremental change, as all of the changes have been since 1994. Personally I think we need to reimagine interfaces to the world around us, even the virtual worlds (VR/AR/MR) around us. Voice input, AI, hand sensing technologies could make for new ways for changing the worlds. The book Daemon by Danial Suarez and his eSpace holds much promise. I have been experimenting with those idea in a new VR world I have been working on for a few years.

jayd16 9y ago
A lot of these technologies are being developed and its pretty awesome. However, I'll argue there's clearly space for a keyboard in the virtual world. If you want to sculpt a model in the real world, you'd use clay and your hands. We can use VR to move away from KBM interaction for things like that. If you want to write a book in the real world you...type it out. Its the best way we know to get text out of your brain and somewhere else. Don't throw the baby out with the bathwater.
- BatFastard 9y ago
  Indeed everything in its place. However I really don't think I would go into a virtual world to write a book, or do any other kind of activity that requires a lot of keyboard interaction. I used to hope for the unlimited monitor space, now I am not so sure how practical that really is.
gervase 9y ago
I think motion sensing is probably the way forward; voice control is intrinsically limiting in most (crowded) environments where computers are used today. Considering humans can ride bikes, drive cars, play instruments, etc. (including typing on keyboards!), I think that indicates that non-verbal, physical interaction is not nearly saturated as a transmission channel. Conversely, it's hard to imagine someone verbalizing "navigate to HN" in a loud open-space office, or "Excel, create a pivot table" or whatever. I think it's fine in private spaces like your home, but in public spaces, you're implicitly broadcasting your activity to everyone around you, which I consider to be a strong negative.

jefurii 9y ago

I could see the hand tracking being useful for people who don't know how to touch type (and have to look at their keyboard and fingers) and who need to type while using a headset.

DoctorMemory 9y ago

I understand Logitech wants to keep selling keyboards and I agree that keyboards are currently the fastest way to enter text. However, it IS the wrong direction. The better direction is what will be opened with tools such as the Vive Knuckles enabling a more complete language of gestures. It seems to me that the better way forward is to stop clinging to an old paradigm that requires bulky equipment that can obstruct the user's volumetric interactions and instead to build off of those volumetric interactions even though there is a cost of a learning curve. I doubt businesses will implement it since they run the risk of alienating current gen customers and thus losing money. I think there will be some VR experiences aimed at younger customers that will implement it and over the next decade we will wonder why we ever tried to bring a keyboard into VR/AR instead of just using our hands.

mncharity 9y ago

If you want to explore this kind of thing, you can mount a camera on your HMD (Vive's is crippled), use a WebVR stack (simple), track objects using visual markers and javascript tracking libraries (jsartoolkit5 and/or tracking.js), and do selective camera pass-through AR. It's crufty, but not hard. EDIT: You can simply use the Vive's camera, with tracking.js color tracking, especially with a small minDimension (number of pixels) threshold. Yellow is good. https://artoolkit.github.io/jsartoolkit5/examples/ https://artoolkit.github.io/jsartoolkit5/examples/ https://trackingjs.com/examples/color_camera.html https://trackingjs.com/examples/color_camera.html ; http://www.keyboard-layout-editor.com/#/ http://www.keyboard-layout-editor.com/#/ https://joric.org/keycaps/#GB-Retro-DSA https://joric.org/keycaps/#GB-Retro-DSA etc

ilaksh 9y ago

How about Logitech makes an affordable effective precise VR glove. Then we can use that for many things besides just data entry.