User Experience Matrix

Imagine this scenario — there are 3 people walking down a real street in Paris:

  1. Real human, just walking along in the real world, wearing AR specs
  2. Tele-presence robot, real, on that same real street, controlled from elsewhere
  3. Someone in the Metaverse / virtual depiction of the real world, walking along the street virtually and seeing what is really there in real time

One can see the robot trundling along, and can interact with it and the person controlling it (via the robot). With their AR specs switched on, they can also see and interact with the virtual person.

Two can see and interact with the real person and the virtual person. The robot can also interact with the rest of the real world, because the robot is actually there.

Three can see and interact with the real person and the robot, virtually. Regular people on the street cannot see this virtual person, but she can see them.

Different types of presence, real and virtual, who can interact with each other in the real world, in real time. They can attend a sporting event or rock concert or pub together.

MapMerge – Where the Real World can be Explored and Used Virtually

Note: I regularly mention Apple or Google as the creators of MapMerge. They are certainly the most likely to do so, but of course until MapMerge is built, we won't know who the builder is. I know it won't be me!

The Short Version: Then and Now

1. Build 3D virtual versions of cities and other places until the whole world is included.
2. Let people travel the world as virtual tourists, and meet with real people in far off places.
3. Create special glasses that let people in the real world see information overlays and virtual tourist avatars. Call them NowSpecs.
4. Use the system to guide, control and authorise robots, autonomous vehicles, drones, surveillance cameras and the Internet of Things.
5. Monetise it by charging for access, charging for virtual public transport, and installing virtual billboards.
6. The underlying structure is called The Then, and the current data that makes it amazing is overlaid, and called The Now.
7. Every smart city wants to be the best. City transportation can soon me twice as efficient -reducing costs and freeing up land. A mapping system, will make the whole picture more accessible.

The Long Version: A Mapping Infrastructure for Virtual People and Robots Travelling in the World. 

The general idea is that many future systems will work better as part of a combined system, just like the Internet of today is a collaborative democracy.

We will have various certainties that may well work together nicely:

  • aerial drones
  • delivery robots
  • neighbourhood surveillance
  • virtual meetings
  • virtual dating
  • virtual shopping

Each of these, once part of a shared platform, will require a map to guide them. One Map.

I envisage a virtual world where billboards will make a higher income virtually than in the real world.

I predict that being somewhere else (virtually) will be more fun than being in your home town’s reality.

As soon as virtual travel becomes possible, modern tribalism will take on a new dimension.

Map Merge is the name for now, but certainly something catchier could be devised. I chose Merge because it is a verb, and people interacting with the map could say they are merging.

Map Merge will (dauntingly) include these:

  • A global standard for mapping the world
  • Street-level 3D mapping of cities and regions
  • Street-level mapping of footpaths/sidewalks
  • Mapping the inside of buildings
  • VR Systems
  • Avatars
  • Air Traffic Control for drones
  • Protection for drones and robots
  • Real-time mapping for self-driving cars
  • WiFi or cell coverage everywhere
  • Google Glass 3.0 or Apple Specs?
  • Cooperation from public transport authorities
  • Cameras everywhere
  • Access to public cameras
  • Software: Making the Planet Seamless in 3D
  • Software: Registration of all Stakeholders
  • Software: A Permissions System
  • Redundancy

Read a fuller explanation here.

Daunting, but Fundable:

Such a massive enterprise will require enormous resources, however that might not be such a big hurdle:

  • Apple and Google have hoarded immense amounts of cash in the region of $100 billion
  • The system can be built one city/nation at a time – the proof of concept can be achieved from a single city
  • The system generates cash via access (utilising micro payments) and virtual billboards
  • The wealth of information possibilities will encourage commercial partnerships with city and regional governments

Interested corporations who perhaps aren’t keen on too much risk can read my ideas for Cautious Corporations.

It Is Inevitable

Not a classic reason for investment, but it seems reasonable that such a system will exist one day, that it is inevitable. Some internet businesses will always be fractured, like media distribution (tv, music, news) where new deals and partnerships are constantly arising.

And some are simply more convenient if all the activity revolves around a central hub. eBay is a great example – no other auction platform will ever be able to compete unless they offer an extraordinary advance in functionality that eBay cannot quickly replicate. Because of the effort required by bidders and sellers, they will naturally invest their energies into the dominant platform only.

There may end up being a variety of systems that access and contribute to the map, but there will be only one map – and whoever controls it will be in a similar position to Facebook today.

To date the 2D mapping scene has numerous players. There is no effort for the user to switch platforms. Their maps aren’t intertwined to any great extent.

However, the first 3D mapping mover has the greatest chance of securing exclusive partnerships that will certainly, one day, use such a system:

  • regulatory bodies (drones, robots, self-driving cars)
  • commercial enterprises (delivery businesses, security companies, retail stores)
  • governments (policing, planning, public transport, tourism).

Because of the futuristic-ness of MapMerge, potential partners will be eagerly forthcoming.

Impact on Every Day Life

People can visit anywhere in the world, and move about it just like in a first-person shooter. They can interact with people who are there. They can meet up with family and friends, have meetings, shop, or be virtual tourists.

Surveillance cameras are already a reality, and the trend is definitely towards more surveillance. MapMerge gives control to the citizens who those images impact the most.

Robotic vehicles (planes, drones, cars, delivery bots, street sweepers, crop dusters, fruit pickers, robocops, maintenance droids etc) need an infrastructure that guides and informs. They will also need to be registered, authorised and/or restrained.

Informational overlays are certainly going to exist, regardless of the delivery system (smart phones, smart watches, glasses, contact lenses, implants etc). A standard infrastructure (with various information providers) would work best.

See:

Getting Everyone On Board Will Be A Mission

Without doubt there will be opposition to a project that has such Big Brother similarities. However, for the most part there won’t be any regulatory hurdles – activities will either be in public, or in private with permission. Certainly Uber faces more acceptance issues.

The seed city (most likely somewhere like London or New York, but possibly somewhere easier to map like Phoenix or Los Angeles)  will need to be a place that people wish to visit virtually.

Here’s a prediction of an adoption path:

  • Local government co-funds the project so they can engineer their world better
  • Public transport, private security, policing, and news services all pay to have access
  • People around the world who want access to family and friends, business or shopping, will pay to do so
  • The radius of the first location expands, and new locations are added, funded by the above
  • The initial success creates an urgency to be involved, and partners will pay more to be involved.

Ideally it will be self-funding from the very beginning. Realistically, proof-of-concept in the first city will be sufficient for MapMerge to be funded by 3rd parties as it rolls out world wide. It would be massively difficult for a competitor to ever catch up. and even harder for them to convince partners to chance a new, incompatible system.

Not Big Brother

Surveillance cameras will be owned and controlled by property owners. A way for the people to control street access cameras is to give people private ownership of streets, like in a gated community.

Permissions are granted by property owners, as to who can visit them in the real world (robots/drones) and people who arrive virtually. Camera owners will share their imagery with whoever they choose, in circumstances they dictate – for example the police can only have access in an emergency or if a crime is committed.

The operator of MapMerge will gain income by allowing access. The more accessible (and less closed) the system is, the more they will earn.

 

Terminology

The Then – a static virtual world, as it was photographed sometime in the past, like today’s Google Street View.

The Now – a combination of the Then (as a framework), with real time information and virtual travellers. Virtual travellers will use a VR system at home or work, and see a virtual world. People in the Real world can experience the Now using special glasses, that show information and avatars as an overlay.

The Real – the real world, in real time.

Merging – combining a real time avatar with a real time place, in the Now.

VR System –  a combination of special glasses and a treadmill for navigating virtual worlds

Avatar – a computer-generated representation of yourself in a virtual world. Virtual travellers will be represented by Avatars – instead of how they are looking at that moment, something most video conferencing and telepresence developers haven’t worked out yet.

Finger Tapping – a means of controlling and navigating a virtual world without using your voice.

 

The Required Tech Already Exists:

3D Global Map – basically an interactive, 3D version of Google Maps/Earth. Google Street View is the closest we have so far, and new imaging technology will be required, along with significant investment. At least they can start with a few major business cities like Tokyo, New York, London, Paris…

3D virtual representations of cities are already being built by numerous businesses, but none seem to desire the creation of a global map.

Improved satellite images combined with enough street shots might be enough to build an approximation of the real world. Google could perhaps pay people for their GPS-tagged photos, as long as there is a compass direction.

Full Virtual Reality System at Home – a headset like Oculus Rift, and an omnidirectional treadmill Virtuix Omni.

3D Human Imaging – already being done by TwinKind for the purpose of creating lifelike figurines. The technology consists of just software and lots of cameras, so being affordable for the home will almost certainly occur.

Customer Kroker has 360 degree photographic scan taken at Twinkind 3D printing studio in Berlin

Twinkind

twinkind_bust

If it can be a figurine, it can be a 3D avatar in the Now.

Fast Internet – some aspects of MapMerge might need to wait until the next level of internet speed. We now have 100 megabit per second Internet in Australia now, and some politicians have complained that we will never need such bandwidth. Map Merge might need 10x that speed to fully utilise it.

The next level of mobile data transfer, 5G, is expected to deliver extremely fast data speeds – perhaps 10 to 50 Gbit/s.

Portable Headsets – like Google Glass, and eventually contact lenses.

Body-Capture Suits – ultimately will be built-in to everyday clothing. For now the technology does exist, but it isn’t very discrete.

Finger-tip Controls – either embedded chips in each fingertip (which I accept that many people will never want, no matter how tiny they become), or super-thin gloves. With the trend being towards people trying to avoid contagious diseases, we might have a glove wearing culture before long regardless.

Drones – already exist of course, and they will become lighter, faster, cheaper and more ubiquitous. One day they will fly and drive.

Interior Mapping Robots – they can either be Roomba-type devices (software can adjust the low perspective, but won’t capture bench-tops etc), flying drones, or proper robots with a variety of cameras. They need to be very cheap and easy to deploy. The entire plan depends on them. Google are developing this exact software for cell phones (called Google Tango), so perhaps robots won’t be necessary.

3D Office Cameras – mounted on each wall, for the purposes of creating room avatars. They might also relay the visuals of the room in real-time back to the virtual visitor.

Internet Of Things – already on the verge of becoming meaningful.

Robots – primarily for delivering items, these will one day be everywhere. They can be personal assistants, shopkeepers, guides… but they will need a 3D map to access, to correlate with what they see.

Telepresence Robots – these exist, but so far they are just an iPad on wheels running FaceTime. I expect these to become so popular for virtual travel that there will be restrictions on their numbers and operation.

See videos about Google Tango (room mapping) and city mapping that are already happening.

Secret Sauce #1 – Income

All of the above is bound to keep developing regardless of any master plan.  But to be truly useful, they will need “one ring to rule them all”. Whoever develops the MapMerge will (my opinion only, of course) be able to own the system. It will be the world’s first and only true global infrastructure. I say only because, like Facebook, I can’t see there being room for multiple, competing systems.

To make it work, there needs to be income:

Advertising – on billboards. Digital billboards will one day be extremely cheap to deploy, and a Google AdWords type system could unite them all globally. Anybody who owns property can install a billboard and earn an income from ads, just like you can on websites today. Technology will be developed to keep track of how well and often people view the ads.

The same billboards will appear in the virtual worlds (both Then and Now). The same people own them. Ads will appear. As Real, So Virtual.

Travelling – there are many ways of achieving this, but I propose that there will be spawning locations around the world that are free. And users will be able to make their way by foot or public transport and look around. But to immediately spawn in a specific location, you will need to pay. There are many examples of where people will happily pay, and first cab off the rank will be business meetings.

Virtual public transport can also attract a fee.

Overlaid Data – people wearing their Google Glass devices will see an overlay of relevant information in the Now, and they will control the levels of bombardment. Many people wish to be informed of deals, like half-price pizza when you glance at a restaurant. These will be paid for by information providers, and/or end users.

Business Meetings – I see these as being an early, big success. Business folk still value in-person meetings, despite the ability to easily and cheaply teleconference. But at some point the quality of virtual will be sufficient to convince enough people to meet virtually. And they will pay handsomely for a service that saves them paying for hotels and airfares.

Regular folk of sufficient means will also meet virtually and pay for it. With time, as more people use it, the costs will come down. I would happily pay $20 to spend an hour with my parents, in their home, but virtually. And they won’t need to vacuum.

Hourly Rate – just being within the virtual worlds can attract a fee, especially when micro-payments take off. If it costs 1 cent per minute to access the worlds, few people will be put off. A smart company would adjust this and adopt the freemium model.

Secret Sauce #2 – Rights Management

The MapMerge concept evolved from imagining how robots will navigate our planet, and the initial idea involved physical devices in letterboxes telling door-to-door sales robots they weren’t welcome. With the widespread use of GPS, geofencing is a much more elegant solution for telling robots and drones where they can and cannot enter. Flying drones will obviously need this data in the form of a 3D map. And the management of the access rights needs to be overseen or run by a singular body.

Take that one step further and you can manage where avatars can venture in the Now. From your private residence to office buildings and rock concerts, you will be provided with access, or denied. All based on your location on a 3D map, and all managed by one entity. It could even apply at international levels (North Korea not letting anyone to cross borders…).

MapMerge vs Cobbled-Together Alternatives

Making your own virtual worlds, and organising virtual visits/meetings will still be possible to achieve privately. But it will take more effort to achieve.

  • If avatars are universally adopted, they will need to be shared by different platforms, or users would need to create a new one for each system they use.
  • If avatars aren’t used, then virtual meetings would most likely use holographs. These will require more sophisticated equipment, will initially be in fixed positions, and will require much more bandwidth.
  • Telepresence robots would struggle to get around without adequate mapping – especially when it comes to steps, kerbs and so on.
  • Revenue would be restricted to access charges, meaning it would be harder to be profitable, and difficult to compete with Map Merge on price.
  • Economy of scale – MapMerge would be able to do everything cheaper because it is covering the whole world. Mapping vehicles, data storage, support and marketing will all be cheaper to supply than a competitor will be able to.

Other Uses for Map Merge

Like the Internet, it will be a case of build it first, and discover new and exciting uses as time goes by.

Here are some possibilities that come to mind

  • Court cases – although in some cases testimony can heard by people from a different location today, imagine the possibilities of every party meeting virtually. It might even be more fair if people aren’t necessarily judged by their facial expressions.
  • Shopping – expect telepresence robots to be used by virtual tourists and virtual shoppers. Primarily the shopping would be part of a tourist experience, for example visiting Harrods in London. I don’t envisage robots visiting Walmart.
  • Police – we are already on the verge of police drones becoming commonplace. Police will love being able to track suspects via an increase in surveillance cameras, and tracking vehicles.
  • Sports – imagine being in the grandstand in the Now. Or imagine trying to hit the curve ball of a star pitcher, in real time, via your VR system.
  • Surgery – a bit of a stretch, but feasible.
  • Rock Concerts – this could be a big success, and a massive income stream for live acts. This call almost fund the development on its own (shout out to Madonna…)
  • Socialising in public – A man, a robot and an avatar walk into the bar on a Saturday night….

 

 

IoT Comms With No Battery?

As usual the devil is in the detail, and I want to know how much data can be transferred how far at what speed.

However, this invention purportedly transmits data by harvesting energy, and doesn’t need a battery or external power source. For the IoT this is massive.

The researchers believe that tiny passive Wi-Fi devices could be extremely cheap to make, perhaps less than a dollar. In tomorrow’s smart home, security cameras, temperature sensors, and smoke alarms should never need to have their batteries changed.
https://www.technologyreview.com/s/600773/10-breakthrough-technologies-2016-power-from-the-air

 

5G Billboard Network

Google has proven that adding some smarts to advertising can be very profitable. The next realm of digital advertising will be in the real world – automated billboards and poster ads in shopping  malls, building walls and traditional billboard locations. This isn’t a new idea, Time magazine mentioned it back in 2010. And Google patented it in 2007, and again in 2010. It has been dubbed “out of home advertising” and Google started taking baby steps in late 2015:

Advertisers might soon be able to buy billboard ads using Google’s DoubleClick technology, which will pull in historical and real-time data signals — including audience, weather, travel information, sporting events, and scores — to decide which creative messages to display, which billboards to display them on, and the best time for them to run.

The idea is that passers-by will see the most relevant ads for the time of day and location they are in. If the passing audience isn’t the right one to show an ad to, then the technology opts not to serve an ad. [Business Insider]

So the idea is there, but to be a major success it just needs some tweaking. It needs to be more widespread, more mobile. It could look like this:

emcoutdoor
*Image source is
EMC Outdoor

If Google/Apple/someone else follow my advice and create a drone monitoring system via 5G, then they’ll most likely take the next step which lets IoT Things communicate with the rest of the world.

Joining the gang of IoT will be digital billboards. Advantages over traditional signage are:

  • ads can be changed at will. Hopefully combined with some kind of colored e-paper technology
  • this allows for a Google AdWord-like system serving ads where they are best received
  • signs can have cams built-in, to monitor foot and vehicle traffic
  • sign owners can be paid according to bids and traffic sighted

The key ingredient that is missing, that makes AdWords such a success, is the measurement of interaction. Well, augmented reality has that covered!
People wearing VR glasses (like NowSpex) will be able to interact with digital signage. It could be as easy as looking at a sign, and saying “sign menu”. They would then be presented with options such as demo videos, nearest dealer, pricing and so on. All viewed in their glasses while they stand in front of the sign, or even on hold until they sit in a drone taxi or VR cafe.

VR glasses interaction can be extrapolated to estimate the interaction of all passersby, because glances from pedestrians and views through tinted car windows is too difficult to measure.

Advertisers can be charged according to VR interactions, and estimated views from people passing the sign. Signage will only exist profitably where people respond to it.

As if that wasn’t enough of a business model, once a 3D environment that mirrors the real world is in place, the real billboards will have virtual counterparts. Double the revenue!

How Soon Will This Be Reality?

Google has already considered it, just like it has considered radio and TV advertising.

Australian company AdBix has come up with signs that respond to the demographics of passersby – although I suspect that local knowledge of who walks by will be just as useful.

Google already measures the demographics of foot traffic via people logged in to Google on Android phones.

E Ink has developed color ePaper. It looks washed out, but hopefully the technology will improve. The ideal is that a sign can run from a small solar panel, which means signs can exist anywhere.

5G mobile data transmission is due to launch in 2020, with more than enough capacity to supply signs with content.

Somebody will try it in the next few years. I predict a form of interactive outdoor signage will be stunningly successful as soon as 2022.

[Originally published at Medium]

How Small Can Drones Get?

On this website the focus on drones has been on security – stopping terrorists and authorising everyone else, and the gathering of data with full permission.

Truly miniature spy drones will certainly exist one day. I have speculated that they might mimic bees or flies to avoid detection.

While these aren’t as small as I have imagined, the Black Hornets certainly tiny and very impressive:

  • Already being used by some major armies
  • Not heard beyond 20m
  • Not seen beyond 100m
  • Returns to base with the push of a button
  • 20 minute flight time
  • Three cameras

The new owners are a thermal imaging company. So you can be located via your heat signature from a device just 20 metres away, and at night and pretty much day as well, you will never know.

Check out this BBC video, and even though the accent is a bit thick,  you’ll get the idea and see how pleased he is with the wee drones: