It isn’t obvious, but autonomous robots are already here. We have robot vacuum cleaners, cars that make decisions, robotic shop assistants and phones that talk back. Late in 2015 we will have Barbie dolls that have conversations with our daughters. Individually they are innocuous, but collectively they represent the start of a new paradigm: devices programmed by humans, yet to some degree unpredictable.

For the most part programmers of these devices can tell you exactly how they will react in certain circumstances. But lately it has become more difficult.

For example, I just asked Siri where the nearest cemetery is (and she understood me correctly) – she gave me the address of the nearest Justice of the Peace. I doubt an Apple programmer could predict that answer.

Autonomy is here, and it is time to add constraints to our electronic helpers. Otherwise their unpredictability will cause situations and problems.

The idea of constraint becomes even more compelling when we consider a future where robots and drones are let loose in the outside world. Currently a robot vacuum cleaner is confined to the home, Siri is confined to a phone, and cars are primarily driven by humans. In years to come they will move outside of our comfort zone: when drones make deliveries and cars drive themselves.

The outside world provides a whole new dimension of unpredictability. Where would a Roomba go if you started it on a highway? How would Siri converse with a thermometer? How will Hello Barbie respond in a conversation about self harm? If the robotic shop assistant follows a customer outside of the store, what happens next?

In the future we will be asking robots and drones to traverse the greater world, and negotiate all the randomness it embodies.

How will a self-driving car react when the car behind rams it? Or when traffic lights stay red and don’t change? Or a sinkhole appears, and to get around it means crossing the median line? What about when multiple situations arise at the same time, each requiring a conflicting response?

Our typical humans like to control their environment. We have systems in place that run our schools, workplaces, communities, governments, and more. We are already controlling people to a large degree. When robots, drones and cars have autonomy, we will control them as well. The future will include a control system for all unpredictable non-humans.

If you accept that bots and drones will need to be controlled, the question is how will we achieve that?

METHOD

I argue that the best form of control will be similar to air traffic control. Planes are susceptible to poor human judgement in a complex situation, so they are controlled. Of course the armed forces are an institution where rigid control is required. Autonomous robots and drones will need to be constantly identified, located and directed. We will be imposing our will on them.

Soon it will be argued that private drones will be too much of a risk. Identification by the public will be impossible, their ability to spy will be unprecedented, and they will become a favoured tool of criminals and terrorists. They will need to be identified and controlled, and perhaps private drones will be outlawed. After all, if all drones are owned and operated by the government, they will be able to provide services of value while also being trusted (at least more trusted than private operators and government operators combined).

It is inevitable that a private drone will cause a dramatic situation or even a fatality, and governments will respond accordingly. Regarding a permission-based mapping system for drones, manufacturer DJI already has software in place that assigns no fly zones to places like airports and the White House.

If we accept that control is required, then we need to determine a suitable system. The scope will eventually be the entire planet, and given the expectation of many flying drones, that means the entire planet in three dimensions.

My suggestion is a global 3D interactive mapping system. At present we don’t have any official global map. The best we have are provided by Apple and Google. Those maps are 2D, with trickery to present some 3D views as well. The next logical step is true 3D mapping.

Aerial delivery drones will need a 3D view of the world. Wheeled delivery drones will need to negotiate kerbs, and find their way about multi-story buildings. They will need to use a mapping system not unlike the worlds of first-person shooter video games.

The system can also be a conduit between the Internet of Things and independent, autonomous devices. Undoubtedly, once such communications are enabled, new and wonderful applications will arise. Perhaps a car will know to slow down because a camera has noticed mother duck and her dozen ducklings approaching the road.

Aerial drones will know where they can or cannot fly. They will be able to enter properties on a permission basis. Autonomous cars will be identified and watched. Telepresence robots will be told where they can and cannot travel – permission obedience will be wired in.

The mapping technology already exists, it just needs to be captured, organised and transmitted efficiently. We already have GeoSim and ReRoll.

WHAT ABOUT HUMANS?

After catering for cars, drones, telepresence robots and the Internet of Things, the underlying mapping system will undoubtedly be used by humans.

Augmented reality is a future certainty, and it will require three things: location of the user, the 3D direction they are looking, and the types of information that interests them. Information access runs two ways – the required information, and the rights to access it. This fits snugly with the location/permission model.

Telepresence robots won’t exist without a human guiding them through the real world. While the human gets to see authentic visions remotely, the mapping system will bed needed to guide the robot, just lie autonomous cars and delivery drones.

Business meetings are expensive when attendees travel internationally, and the dream of virtual attendance has yet to become an acceptable and affordable alternative. Lifelike avatars that appear to be in the meeting room itself (rather than seeing a face on a monitor) could be the breakthrough.

Armchair travel has been around since the invention of the printing press, but never before have we been able to feel like we are actually there. People will test-drive holiday destinations before physically travelling.

Education and training are an important part of the employment process. While the armed forces have used virtual realities for training for a while now, this can now be used by less substantial businesses. For example, a forest ranger can virtually visit key places of their jurisdiction, instead of travelling for weeks in the real world.

In the justice system, juries and judges will have new and interesting ways of viewing a crime scene – from the comfort of the courtroom.

Perhaps the biggest human interaction with the system will be social. More than ever people are separated from friends and family by distance. Places are held dear, such as bars and restaurants, parks and playgrounds, family homes. To be able to interact socially in a meaningful place will be a driving force towards the adoption of a universal mapping system.

Entrepreneurs will aim to gain extra patronage from virtual presence. Obvious scenarios are attending sporting events and concerts. Access to exclusive scenes can be sold to the highest bidders – imagine attending a famous nightclub, or a party at Playboy Mansion, while sitting at home.

ROLLOUT STRATEGY

Cities that are business centres, popular with tourists, compact, and open to the future could be shortlisted.

Starting with the Central Business Districts of the biggest and most modern cities makes the most sense. They have the most inter-city business meetings (more potential for virtual meetings), CBD districts are compact, they have businesses willing to invest in future-tech, and they tend to be visited by tourists.

Candidates to get the ball rolling are a combination of financial hub, tourist destination and modern technology: Seoul/Songdo, Barcelona, New York, Tokyo, Shanghai, Singapore, Amsterdam, Cairo, Dubai, Kochi, Bristol and Malaga.

 

Armchair travel is already a thing, so imagine being able to virtually travel a far-off land from your armchair in 3D video? If the 3D mapping already exists, we have a framework for virtual tourism.

Give those virtual travellers an avatar and suddenly we have a parallel world populated by real devices and virtual people, all interacting with the regular world we know today.

And for business meetings, there will be a distinct advantage for businesses that offer live, VR meetings. That in-person meeting involving flights and timezones can be achieved much more efficiently.

I envision the rollout to go like this:

3D mapping companies contract with major cities to create 3D maps of their cities. Municipal leaders can be convinced that owning the spatial data of their city is important, and they need to be proactive. The uses for the data can include:

• accessibility information for the disabled
• intel for emergency services
• visualisation for administrators
• identification of required roadworks and maintenance
• proving code violations
• managing traffic / crowd control
• optimising public transport
• location-based mass communications
• crime detection
• augmented reality for tourists

Then gamify the proposition – how they can be the first/best city to have such a system, or how they can share data points with other digital cities. Ultimately the smartest cities will be alluring, fashionable and desirable as regular citizens become more tech-savvy.

Add surveillance cameras into the mix and the police get a whole new level of intel, as does whoever controls the cameras. I envision societies where suburban streets are private, and the property owners control the surveillance, not the government. Likewise commercial strips can have cameras owned and operated by the present businesses.

Access to surveillance footage will be by explicit, pre-determined permission. The virtual infrastructure is now a 3D map and a permission based system of information access. Throw in the Internet of Things and location/data/permission makes a lot of sense.

Returning to drones and robots, how will they be controlled when traversing the greater world?

DRONES and ROBOTS

Traditionally the identification of vehicles has been visual – writing on the exterior. We have cameras that record the licence plate of a car, but they are fixed cameras on busy highways, aimed at exactly where the licence plates would appear. Flying drones are much more difficult to identify visually due to the distance they might be from cameras, and their three dimensions of free movement.

A simpler system is used on many toll roads today. The vehicle has a device onboard that projects its identification to local receivers. When checked against a database in real time, it can be determined if that vehicle has permission to be there. On toll roads, a vehicle without permission can be required to pay a fine. With flying drones, a lack of identification and permission could result in it being intercepted.

Given that drones are remotely driven, and potentially unidentifiable, they could easily be utilised by criminals and terrorists – a strong reason for a high degree of identification and regulation.

Requiring robots and drones to identify themselves, combined with 3-axis locating, means that remotely or autonomously driven vehicles can be accounted for. Throw in licensing and methods for taking down illegal machines, and we have a safe and controlled environment.

 

MANOUVERING

Wheeled drones will most likely travel on roads and footpaths, meaning interaction and potential collisions with humans and the vehicles we drive. Consequently the drones will need features such as:

• visibility to us – they need to be of a reasonable height, say 5 feet, and not too thin when seen side on
• courtesy – giving way to humans and drones that are on more important missions
• efficiency – if their way is impeded for too long, alternative routes need to be determined
• safety – collision avoidance is important for both parties that could be involved
• redundancy – the ability to function if their data connection with the mapping system is severed
• security – thwarting theft and vandalism, or at least making sure that culprits are identified

CARS

Recently there has been a lot of talk about autonomous vehicles, as if they will replace all current types of road transport. Mostly the discussions ignore how integrated people are with their cars, and how much they love controlling the speed and power of their second biggest investment.

Autonomous vehicles will initially be most successful as public transport, replacing taxis, in CBD locations. Downtown environments are best suited to early adoption of self-driven vehicles, because of the lower speeds and greater density of need.

Robot taxis will certainly be most efficient if they all belong to a single collective. That collective will rely on spatial data, perhaps integrated with other information sets, such as traffic flow and accumulations of humans. A single system would work best. Outside of the government run robot taxi infrastructure, the only other factor is the supply and maintenance of the vehicles. Given that there are no human drivers, and efficiency comes down to software that can be replicated across cities world-wide, there is little need for capitalist competition. Cities can own and control robot taxi systems and do so efficiently.

HURDLES

Radical changes to the operations of society always invite feedback and fightback. Through popular media we have been conditioned to be afraid of Big Brother and Skynet/Cyberdyne Systems. And opposition will occur naturally as well. Yet we have seen with Facebook that people are willing to concede privacy for utility.

The other major hurdle is the cost of implementation. Starting with major cities and including the municipal authorities is a likely path. But if a private business ends up building the system solo, the infrastructure investment costs would be potentially be so high that few companies could afford such a gamble. Potentially only Apple and Google would have the resources to bring it to fruition.

ROOFTOPS

The concept will almost certainly begin in the downtown areas of major, modern cities. This of course means skyscrapers, and most skyscrapers have a roof.

Roofs of high-rise buildings are typically under-utilised. In the near future those roofs could become transportation hubs:

• flying drones deliver cargo from inner-suburban distribution centres
• cargo is transferred to wheeled drones on rooftops
• rooftops are also solar-powered recharging stations
• wheeled drones reach the street via elevators down the outside of buildings

Alternatively they could use internal stairs and elevators, however most buildings won’t be designed for this to be efficient. External elevators are very possible as long as the drones and cargo are of a light weight.

PERMISSIONS and BUILDINGS

It isn’t difficult to imagine a city building  a 3D map of all public spaces. But what about private property? For robots, drones and virtual humans to traverse all of a city, they will need to be able to access private property.

First and foremost, current law and restrictions will still apply. Yet permissions will usurp them. Owners of private property will allow drones, robots and virtual humans to enter their premises for the same reason they allow regular humans to do so – it is beneficial to society and commerce.

The boundary between public and private will a challenge. The same system will need to supply data for both. Non-human entities will need to see their whole world as a seamless, integrated system. Private property will provide access to the mapping system’s data gathering machines. In return they will get holographic avatars attending meetings, and robots delivering pizzas.

WHEELED DRONES

There will be three varieties of wheeled drones, defined by where they are allowed to travel. The first will be intra-building, limited to the clearly defined boundaries of a singular location. We already have these to some degree. The second will be allowed to traverse all public spaces, primarily for delivery purposes, but also for public functions currently provided by human officials – such as a police officer giving directions to a tourist. Yes, a rare thing today, but easily achieved with the robots of the future.

The third variety is a hybrid of the first two – it can go anywhere it has permission, and can access indoor and outdoor mapping. A high-level of purpose and trust would be required.

For efficiency of deliveries (such as a pizza being delivered by a drone), open space drones will need to exchange packages with internal drones that know their way around a building, and how to best interact with its inhabitants. The exchange would take place in the main lobby of the building, or outside the ground floor elevators. It would be most easily achieved with a container system (analogous to shipping containers) where the receiving drone swaps an empty container with one containing an item to be delivered. Perhaps the dimensions of such an ubiquitous container is worthy of considerable debate…

REVENUE

There are a number of ways that the mapping system can be funded, and given that the expected costs of implementation will be historic, the funding methods chosen will be critical. A subscription model for devices is easy to predict, but what about humans?

PAYG – Pay As You Go, a direct subscription model, has numerous benefits and potential angles. Commercial use could be levied while public amenity could be free. Human augmented reality users could pay by the amount of data they consume. Or informational aspects could be sold by auction, not unlike the present day auctions of data spectrums. For example, accessing weather data, or pedestrian counts.

Advertising – the future of billboards is a combination of Google AdWords and large digital screens in the real world. Businesses will bid on real-time auctions to have their ads displayed in the places where they get the best response. The billboards can appear in the virtual world as well. To ensure the virtual world is not overrun with advertising, billboards could be restricted to spaces used by their real-world counterparts.

Transportation – virtual travellers would love to be able to instantly arrive at the precise destination of their choosing. A levy could be placed on such direct access, with the free alternative being travel by foot – where you literally walk or at least travel at the speed of walking. In-between would be paying modest amounts to use virtual public transport. It will take some time to arrive somewhere, but you will be able to see the virtual world around you – not unlike looking out of the window of a bus or train in the real world today.

Would people take virtual public transport? In Norway a 7-hour train ride in HD video was a big hit for armchair travellers – establishing a new genre of entertainment:
https://en.wikipedia.org/wiki/Slow_television

Levies and Fees – just like the Internet, initially the mapping system will be used by enthusiasts and public institutions. In the long term, private business, and business models, will dominate. Fees and levies could be imposed, not unlike those of the banking industry, where the infrastructure is funded by the corporations using it, and ultimately by the consumer.

Public Funding – it is possible that, just like the Internet, the mapping system evolves on the back of public funding. The major problems with this are:

• rollout time (slower)
• inter-operability (more complicated)

(Not) Virtual Real Estate – a virtual world will work best if it mirrors the real world physically and in terms of ownership. Attempts to monetise the virtual world by selling virtual real estate will most likely fail, as could be said about the ground-breaking Second Life.

Reduced Road Use – autonomous vehicles will be more efficient in many ways – quicker to respond to traffic movement means less traffic jams, vehicles will likely take up less space, dedicated lanes for different vehicle types and accurate driving leads to narrower lanes. These suggest less road width required, which in turn means the potential for repurposing parts of road, potentially reaping billions for major cities.

In downtown Melbourne there is Lonsdale Street that is a six lane road at places. Halving that road would alone realise hundreds of millions of dollars of real estate…

TECHNOLOGY (EXISTING/NOT)

Remarkably for a futuristic concept, the technology is mostly in place already. 5G wireless communications have been developed, and are awaiting rollout. The long-term trend is toward seamless connectivity for all. Wearables for augmented reality exist, but need to become less cumbersome. 3D mapping of cities has begun. The next generation of budget drone-craft will be able to carry payloads. Augmented reality, permission-based geo-access, and a global mapping system will certainly be solved with software without any major hurdles foreseeable. And the trend towards surveillance cameras becoming ubiquitous is obvious.

Interior 3D Mapping – Google’s Project Tango enables the 3D mapping of interiors using just a smart phone.

Exterior 3D Mapping – a number of businesses are offering this service. A good example is GeoSim.

Video Games using Real Environments – an example is the game ReRoll where its 3D worlds are actual maps created by drones flying around real environments.

5G Mobile Data – expected around 2020, mobile devices will receive data at a rate similar to the broadband wired devices of today.

Telepresence Robots – a number of companies offer these, but at present they are basically a merging of iPads and Segways. The technology is not a problem, but design and public acceptance need work.

Self-Driving Cars – also known as autonomous vehicles, are the subject of significant research and are considered inevitable. Even so, there are technical and legal hurdles that have not yet been resolved.

Delivery Drones – Amazon and other businesses are predicting a big future for flying drones, although improvements will be needed in battery efficiency before they are able to carry decent payloads efficiently. Wheeled drones are easily achievable with current technology.

The Internet of Things – gaining the most attention are products from companies like Nest which connect multiple devices in a shared environment. Of more significance will be the simple data providers that will become ubiquitous, like street lamps and surveillance cameras. Less obvious devices such as dirty bomb detectors and clean air measurement are already in use.

VR Glasses – two different technologies, Google Glass and Microsoft HoloLens, already provide the required technology. In the future these devices will become smaller and more powerful, and more acceptable to the general public.

VR Entertainment – while VR games will be grand success, other uses for the headsets will evolve. Exploration of the real world is an extension of the current uses for Google Maps and their Street View function. Streaming of live sporting and entertainment events to VR headsets is not far off.

About all that has yet to be implemented is the unifying system described here.

COMMERCIAL ROLLOUT

Ideally a company with tech clout and immense resources (Apple or Google) would recognise the potential, work out the cost and revenue figures, and get first-mover advantage. In that scenario, starting with a handful of large cities would help them achieve earnings the quickest.

The 3D-mapping process, if the machines are slow and cumbersome, would require an agreement with local authorities. Public/private partnerships are becoming quite popular, and progressive cities would embrace the idea of having access to such data for free. Once a few major cities are on board, others will be keen to join in. Businesses will lobby them for the ability to have more immersive virtual meetings than video conferencing can provide.

Without such singular vision, the rollout will take a crooked path, and involve a confusing variety of stakeholders. These differing threads would be combined at some stage, a merging of maps:

• public transport
• autonomous vehicles
• the Internet of Things
• 3D maps of cities using different technologies
• virtual meetings
• augmented reality
• augmented reality monetisation
• VR systems in homes and businesses
• wearable technology
• drone deliveries

Each will use maps to some degree. Each will have multiple, competing technologies. Ultimately the laws of synergy will prevail, however the process will be long and complicated, and the ultimate controller will either be public/non-profit, or an organisation who has leveraged buyouts to achieve a modest return on investment.

If the concept is inevitable, the question is how are we going to get there?

ABOUT THE AUTHOR

Rob Skelton is a futurist who focuses on large, global scenarios with extraordinary  consequences. He introduced the world to the 2012 doomsday meme (under the pseudonym Robert Bast), and is currently looking for evidence that we live inside a computer simulation.

Further information regarding the global mapping system is at mapmerge.net.