码迷,mamicode.com
首页 > 其他好文 > 详细

演讲稿: 如何使用增强现实技术提高应用的用户体验

时间:2019-10-12 20:23:30      阅读:146      评论:0      收藏:0      [点我收藏+]

标签:mos   before   rtu   role   win   screen   graph   sse   sample   

With the accellerated evolution of mobile hardware, application developers tend to reconsider the importance of the Augmented Reality technology ( called AR for short in this article subsequently ) in regard to satisfy their customers better with enhanced user experience.

What is AR? See one explanation from Wikipedia:

Augmented reality (AR) is an interactive experience of a real-world environment where the objects that reside in the real world are enhanced by computer-generated perceptual information. AR can be defined as a system that fulfills three basic features: a combination of real and virtual worlds, real-time interaction, and accurate 3D registration of virtual and real objects.

技术图片

This blog will not explore the possible scenario where AR can play its important role. Instead, Jerry will introduce a kind of development approach which enables developers to generate applications leveraging AR without deep knowledge on Computer Graphics.

This approach consists of pure JavaScript technical stack:the combination of React-Native plus ViroReact.

Jerry has introduced how to use Cordova to package a hybrid mobile application in the past:

技术图片

And compared with Cordova, React-Native has quite different design idea: while Cordova can easily allow existing web application to run under different mobile platform at the cost of some performance loss, the application built by React-Native can behave more like a native mobile application.

For the concrete comparison between these two technical stacks, please refer to this medium article.

ViroReact,a platform for developers to rapidly build native cross platform AR applications using React Native. Its website provides many step by step guide by following which, you can soon get a running Hello World application with AR enablement.

技术图片

Although just a single declaration code to introduce the dependency of ViroReact to your React-Native application as below, it‘s just the tip of iceberg.

技术图片

When we enter the root folder of application with react-viro declared as dependency in its package.json and perform npm install, lots of artifacts are automatically downloaded into the node_modules folder:

技术图片

Expand folder react-viro and we can find two separate folder, android and ios, which contains corresponding library codes for each platform. This mechanism implies how ViroReact achieves the cross platform enablement of AR feature. And what‘s the meaning of arcore_client under android folder above?

Before we go deep into the source code, let‘s acquaint ourselves with some prerequisite knowledge. In ViroReact website, it is documented that both ARKit from iOS and ARCore from Android are supported.

技术图片

In this blog I choose Android platform which means I will concentrate on ARCore. Still remember the arcore_client folder mentioned previously?

Let‘s first see some real examples, developed by my colleague Wang Leo:

https://youtu.be/05A0bIiVouE
https://youtu.be/Rt163cQ_fbg

Amazing, isn‘t it?

Below are steps how to develop an application like that and make it running in your Android phone.

As the first step, ARCore should be installed on the Android phone.

ARCore is Google’s platform for building augmented reality experiences. Using different APIs, ARCore enables your phone to sense its environment, understand the world and interact with information.

技术图片

Check the supported device list from Google website and install ARCore into your device according to corresponding instructions:

技术图片
技术图片

In ViroReact website there is a nice tutorial for beginners( just as our SAP UI5 tutorials):
技术图片

Just follow the steps there and finally you will get a Hello World application with some computer generated virtual objects positioned within your real world:

技术图片

In fact the AR car demo by my colleague Leo is also based on this tutorial, so I will introduce the delta part Leo has added on top of the Hello World tutorial.

In React-Native package.json file, name the project as "ViroSample" and declare the dependency of React-Native and ViroReact as below:

技术图片

Execute the following two command lines:

  • react-native start
  • react-native run-android

And you can find the corresponding generated Java source code for Android platform under android folder. The most important one is MainActivity.java, where the project name defined in package.json is injected:

技术图片

And in MainApplication.java, the entry point of React-Native JavaScript module is injected in Android callback getJSMainModuleName:

技术图片

In my opinion the common process of ViroReact development consists of three major steps: Match, Replace and Augment.

Match step

As developers mix computer generated virtual objects with real world, they must decide which places in real environment could be used to hold the virtual objects and tell AR application the exact position. In ViroReact based AR application this is achieved by the tag . See one example below.

技术图片

As its name hints, this tag declares a marker which guides ViroReact with exact position where the virtual object should be put on. As long as end user opens device Camera and detects a real object in the real world which is exactly the same as the object represented by the marker, we call this process as "matched" and ViroReact will automatically replace marker with another prepared virtual object ( will introduce soon ). As long as the marker is matched by some object in the real world, the replace process is automatically done without any additional coding from developer.

A marker is a placeholder in the application and could have various types, and of course ViroARImageMarker has Image as marker type, the image marker is indicated as name "logo" defined in line 44 above.

In the above example, the marker named "logo" is created by the following code:

技术图片

The input of createTargets API is an image file stored in project folder:

技术图片

This is reason in the youtube demo video that first Leo displays this image in his screen, then scan it via phone camera, and the image marker is replaced by the prepared car 3D model.

Replace step

As mentioned earlier, there is nothing to do from developer in this step except providing a 3D model to be replaced by image marker.

ViroReact provides the tag which represents the 3D car model consisting of a pure model ( .obj file ) and a series of material textures.

技术图片

Below is the obj file for this car demo, opened by 3D Design software:

技术图片

In Paint 3D software, we can drag it and have a 360 degree view on it. You can see the dynamic effect from this video:

You might think that the car model has no color on its surface and doesn‘t look not fancy at all. So do I. That‘s why we need to prepare texture as well.

In the project folder six different image files are stored there as raw material for model texture.

技术图片

Colorful materials are created using those texture images files, and finally generate the glossy car model in the youtube video.

技术图片

Augment step

Finally it‘s time for we developers to play with those generated virtual car model by JavaScript code. Since now we have six colorful textures at hand, it follows logically that it could be nice if the car in the AR application could change color every time it‘s tapped in the phone.

A piece of cake: register a function to onClick event handler:

技术图片

And change the binding field for texture property accordingly:

技术图片

Check this video below about how the Tesla car model in the application can change color once pressed:

Hope this blog can give you a basic understanding how to develop AR application using React-Native + ViroReact, thanks for reading.

技术图片

要获取更多Jerry的原创文章,请关注公众号"汪子熙":
技术图片

演讲稿: 如何使用增强现实技术提高应用的用户体验

标签:mos   before   rtu   role   win   screen   graph   sse   sample   

原文地址:https://www.cnblogs.com/sap-jerry/p/11663503.html

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!