16. June 2022

Lens Studio – AR with a snap

Snap.Inc as a company is most famous for their main creation "Snap Chat" and since many years, millions of people use their app daily. A few years ago though, Snap.Inc released their application Lens Studio and gave its users the chance to make their own lenses. Through free templates and an easily understood interface, it enables creations from everyone. There are already thousands of users, beginners, freelancers and companies alike, who created and published lenses.

Lenses can be as little as a 2D object placed into the cameras view and as much as a whole game that can users can play through a filter. These lenses can be used in an impressive matter though, as they have the ability to combine reality with virtual assets.

Augmented Reality made easy

The possibility of creating AR advertisements through these lenses, might be the most exciting feature for businesses. Yet basic and fun lenses can already be achieved by nearly anyone taking their time to do it. Snap.Inc's application tries to be as easily understood as possible, and to be honest, even someone without any experience can create a simple lense. Even though basic lenses are easy to create, the possibilites they offer when done with expertise are amazing.

BenjiX
Try on the BenjiX shoe with this example lense

Workspace

As soon as we open Lense Studio, a interface appears that many experienced developers already know. The Layer structure on the left tells us which cameras are active, what kind of assets we are using and any thing else that is part of our scene. The main screen can either be a 3D workspace, in which we can manipulate our models or a node block, that visualizes the code behind the lense. Each Lense you create can be of a size of up to 8MB, which is obviously not a lot, as many pictures you take with your phone already take up 6MB of space. Inside of Lense Studio, you can either work with 3D assets or interact with the nodes, describing the scene, directly. Inexperienced users can simply start with one of the many premade templates.

3D workspace and nodes can be manipulated side by side

Lenses for Marketing

There are already some companies using Snap Lenses to present their products to their customers. Anyone can simply start the application and see at how they'd look wearing them. From glasses to shoes, you can try on near anything through some AR lenses. Even though the market is definitely picking up on this, there are still many chances for improvement.

AR goes beyond

A lense can let users try on things, change their face or add sparkling effects, but it can be much more than that. Using Lense Studio, we at weltfern even already created an interactive game. Although the game was not allowed to be larger than 8 MB at that time, we managed to create an interactive experience with 2 cameras. Today, thanks to the Snap AR partnership between weltfern and Snap.inc, far more complex applications and games are possible.

With the help of the node system and some experience, you can create immersive lenses that will be far more than just a filter! Through programming, these can push the limits of AR.

What now?

The first step into the world of Snap Lenses is an easy one, yet as you will notice fairly quickly, there are many things to explore and do. There are many templates that anyone can use for free and create AR experiences. Even though some lenses already are impressive, we at weltfern see the vast potential for Augmented Reality. Since the beginning, we have been working extensively on AR possibilities including call-to-action and integrations of new features directly with Adidas. Our expertise even earned us the official title of Snap AR Partner. With our experience, we can help you with creating an outstanding experience for your customers. May it be for sales or just for presentation, augmented reality is a great tool to present your product.

15. June 2022

Metaverse – What and How?

Nowadays you will be hardpressed finding anyone that has not heard about "the" metaverse. Lastest after Meta (formerly Facebook) decided to rebrand their whole company, in order to embrace metaverses fully, it stands out of question, whether metaverses are realistic. Meta did not invent metaverses though, nor is their creation the first or only metaverse on the market. For this, it is important to answer, what exactly the metaverse is. At the same time it's quite interesting to ask, how a metaverse exists and how we can interact through and with it.

What is a metaverse?

The easiest way of describing the idea of a metaverse is to describe it as an alternative and virtual reality. The basic idea behind it is to have a virtual place in which we can interact with one another. This can go from as little as a roleplaying game up to a virtual meeting room. Through the use of virtual bodies, called avatars, we as users might even place ourselves inside such a space. As the virtual world gains importance and influence daily, metaverses open up like doors granting access in a whole new way.

How does a metaverse exist?

Next to the creation of such a metaverse, we have to think about how we gain access. As forementioned, a metaverse is but an interface for people using the virtual world. The questions that really matter might be which files we can use to create such a space? And just as if not more interesting, how can we give users access to said files and assets? On the weltfern newsblog, we have articles and insights on some exciting background around metaverse content creation. Feel free to take a moment to read up on the topic as needed. Nowadays it is very possible to create such a world, yet the workflows still present to be difficult.

A metaverse needs to be versatile and normally has to be created with a multitude of applications. Since businesses tend to try to reach as many customers as possible, such an universe probably needs to be multimedia capable as well. As realtime is needed to give instant access, Pixelstreaming and similarly expensive external providers have to be used as of now. Because of this the hosting of metaverses oftentimes is still quite expensive. We do want to take a look at some applications that could prove to be useful for metaverse creation.

Unreal Engine

We discussed the usefullness of Unreal Engine in the past and it's an incredible application with a plethora of usecases. Yet it might not be the most useful for the creation of a metaverse. It is true, that artists can use it to place and create most scenes they need for a metaverse. Another valuable fact is, that companies can use UE for working in teams. The engine is capable of reading many file formats, giving a multitude of workers access to a single scene. The reason UE5 is not yet ready and useful for the creation of metaverses, doesn't lie within the engine itself though. At the moment, businesses more of struggle with bringing their creation to their customers. The only real way to do this for now, is Pixel streaming or an equally expensive 3rd party extension, as WebGL does not yet exist for UE5.

Omniverse

NVIDIA Omniverse™ is an extensible platform for virtual collaboration and real-time, physically accurate simulation. The true use of such a platform, is the ability of creators, designers, researchers and engineers to come together and work on a shared virtual space. Omniverse uses .usd files to reference assets created in a multitude of applications. Because of that teams are able to work on the same project without having to worry about file conversion.

The question now is, how does Omniverse differ from UE5 and why is it more useful for the creation of a metaverse than Unreal is for now. Well the main difference are the file sizes. While UE5 can import assets, Omniverse only references them, which makes the final size a lot smaller. The main reason Omniverse can do what it does is the file format .usd and .usad it uses.

.usd what does it do?

Universal Scene Description (USD) is the first publicly available software that addresses the need to robustly and scalably interchange and augment arbitrary 3D scenes that may be composed from many elemental assets.

Created by Pixar .usd is short for "universal scene description" and does exactly what it sounds like. The software uses .usd to describe assets, placing those description in a scene or relating them to other assets. To visualize this file system, omniverse uses a layer system. Each .usd is put on its own layer, which are ordered in a hierarchy system.
Because assets are only referenced though, they can't be transformed inside a scene. Each creator can work on their original asset inside their respective applications. Each transformation (position, size, etc) is noted on the layer, still as a reference to the original asset. This way a team can work together to create something complex and finished.

Sources

Unreal Engine 5

https://www.unrealengine.com/en-US/unreal-engine-5

Omniverse

https://www.nvidia.com/en-us/omniverse/

Images

All images and screenshots were taken for research purposes only, neither them nor the projects are for comercial use.

13. June 2022

Realtime content with Unreal Engine

Recently Epic games, the developers of Fortnite, one of the most successful games, released the version 5 of their Unreal Engine (UE5). The weltfern team has worked with this game engine from the beginning. Even now, we are creating high-end software and simulations for Virtual Reality and Augmented Reality. Back in the days, before it was the hype, we used Unreal, to create our own metaverse "white-plane". Today more and more companies create applications and "metaverses" with Unreal Engine. But why Unreal Engine and what is it all about?

The ideal way to real-time

If the idea is, to create something, which can react and render anything directly as it is happening, we are talking about realtime. Oftentimes we are used to beautiful imagery, animations and presentations, that are set up and rendered for a long time. Using realtime, we build a scene and then can move through and manipluate it instantely. It does not matter whether its for the B2B area or directly B2C in sales, realtime offers an intense opportunity for interaction. Realtime applications give direct access to worlds and assets, using either complex avatars (metahumans) or as an interface let users percieve them through their own point of view. When talking about realtime, there is no getting around talking about game engines either. They are, what is most commonly used, for the creation of realtime applications.

Getting started with game engines

The concept is ready, the content research successful, the client on board and the budget is there. Together with a team of interface designers, realtime 3D artists and developers we start creating a level, an MVP or directly the first beta version. But beware, development and especially bugs can become time killers.

Focus on production

The most important thing in the production of real-time content is not to lose contact with the product. Whether in the agency, at client site, or directly on the set, the awareness of how content interacts with the asset is essential.

Advantages of Unreal Engine / UE5

There definitely is an ideal scenario, but it requires consideration of all the parameters (Input, file size and prefered output). A client that wants to shoot a commercial and already has planned a fixed budget for it, being able to quickly recieve a final product sounds perfect. For companies or teams that deal in one product exclusively, the choice of developing it completely with only one engine is surely tempting.

In the beginning, nothing is easy, especially not with game engines. These engines are basically always work through the use of logical connections. So there is always coding / software developer work required! In the case of UE, the beginning is made significantly easier by the visual scripting possibilities through blueprints. Nevertheless, solid basic knowledge in 3D is recommended, not to mention C++ knowledge, at least, if you plan to program anything serious.

How to use Unreal Engine 5 (UE5)

Our expert team of experienced developers has summarized these key features about programming in Unreal Engine:

Rapid Prototyping

  • Blueprints are a great tool for rapid prototyping. It is extremely flexible and powerful as it provides the ability for designers to use virtually the full range of concepts and tools generally only available to programmers. Even our most experienced C++ developers like to use them.
    • Blueprint-specific markup available in Unreal Engine's C++ implementation enables programmers to create baseline systems that can be extended by designers. (Unreal Engine Documentation)

Serious development of applications and games

  • As soon as we get deeper into the developement of high-performance applications, it is crucial to understand the workings behind the engine.
    • An understanding of C++ is the only way we can understand exactly what is going on and intervene in case of doubt.

Do not forget: Blueprints are also C++ code, visualized using a node-based interface though.

The most significant features in UE5

Get Realtime skills with a Game Engine workshop

Take the chance to build simple content creation skills in your team! Everything necessary to implement high-end environments in UE4 and UE5 or other engines like Unity. We at weltfern offer beginner and advanced courses that will leave a lasting impression and worthwhile experiences.

The unreal dream of UE5

Limitless polycounts allow you to use models of any complexity level in your scene. This of course is very tempting for many productions and therefore already in use today! Always be aware that all efforts, pipelines and integrations that make Epic exceptional are driven by the goal of using Unreal Engine as the "end goal".

What does this mean for CAD conversion?

No limit on polygons in UE5, datasmith importers and many other improvements, could sound like solution for the most issues with CAD conversion. Nothing is ever without issues or questions, so we dedicated a whole article talking about whether UE5 is a gamechanger for CAD. In short, every conversion, no matter how good and precise has its weaknesses and opens up new pathways and questions. Not only does the CAD model need to be flawless, there still has to be done some preperation.

Disadvantages using Unreal Engine / UE5

Especially in the exchanges with customers, we often meet mistakes and flaws in the ressources delivered. These flaws can not only be time consuming but also budget intensive. Make sure to properly inform all parties involved of any possible complications.

Multiple formats are essential

Thanks to the Internet, today's concept and advertising has to be multimedia. Therefore companies have to cover multiple formats, which means a multitude of prepared sources are necessary.

But wait, if we use high-poly models in UE, how do we get them out of the engine? At latest this step makes it obvious, that a whole new area of problems arises. While this makes one workflow simpler than before, it creates another at the same time.

Large files could prove to be just as big of a problem, as they make the application quite huge as well. No customer likes to wait for several hours for a download, so you have to find a solution from this point on.

How about UE5 for your own metaverse?

Lyra (UE5) - A free headstart into your own game

If you have the budget for massive and expensive servers and pixel streaming, go ahead. Otherwise the problem lays not within UE5 itself, but rather the missing interfaces. The engine offers you a boatload of possibilities to create attractive metaverse applications and assets. It even comes with a lot of starter content, with which you can easily build your own world. Yet the real question is, how do we get our creation online and running?

Use UE5 and profit?

Anything new always gives hope for improvement and the chance to profit. As for UE5, calculating the key factors in your production and accurately evaluating the costs, will probably save you a lot of time, which equals money. The ability to use a pipeline where the number of polygons and the complexity of the geometry are practically irrelevant is an incredible opportunity! Even if some features might not be ready yet, they are sure to evolve, so be patient and stay informed.

What to do now?

I'd recommend you go ahead and experience UE5 for yourself. The engine delivers improvements and new possibilities that are worth it. If you are either missing experience or time, it might even be a great idea to contact us at weltfern to get an insight through our expertise. A lot is possible with UE5 and an expert at your hands, don't hestitate to get in touch with us for any question.

Sources

Unreal Engine 5

https://www.unrealengine.com/en-US/unreal-engine-5

Images

All images and screenshots were taken for research purposes only, neither them nor the projects are for comercial use.

3. June 2022

Is UE5 a gamechanger for CAD in realtime?

The Unreal Engine 5 is using "Nanite" to virtualize geometry and achieve detailed rendering of high poly objects. The constant improvement of technology is already changing classic workflows and will influence more and more fields. All of a sudden, the lowpoly pipeline is no longer relevant. Artists can now insert complex geometries into the engine and edit dense 3D meshes without the usual effort and time requirements. This will lead to the need of visual developers and the evolution of game artists, that are used to the creation of super compressed models.

After the new Matrix movie and the playable games made with UnrealEngine 5, the hype is definitely real. As more topics regarding UE5 are being revealed and researched, we took a look into the implementation of CAD models into UE5. As anyone, that had to deal with this issue, will confirm, this has not been without problems in the past. Through the use of datasmith, UE5 promises a more fluent workflow, no matter which program the CAD model is created in. But is UE5 a gamechanger for CAD in realtime? May it be Rhino6 or autocad, the weltfern team researched the possibilities and has some exciting insights.

CAD in Realtime

Computer aided graphics (CAD) is the industry standart for production. Manufacturer requires a model in .step or .stl format from their engineers or product designers. In fact all volume and strength calculations necessary to produce any product / part need CAD files as input (even 3D printer).

However, as soon as this product is expected to make a profit and needs product renderings, we will need it in polygons. Because of that everyone faces the issues with CAD conversions. Graphic cards render polygons and real time engines need conversion to visualize properly.

Creation of a high-end CAD conversion for automotive (using augmentable)

UE5 presents the possibility of making this process unneccessary. Their offer of using datasmith for the implementation of CAD models in UE5 is on of the engines key features.

With the internal tool Datasmith, Epic has created a very powerful tool (including plugins for all major 3D tools) that can even read runtime CAD formats covert them on the fly and load them into the engine. While the workflow becomes easier and faster, we cannot forget though, that the imported models are still delivered by the industry.

Import quality depends on the source

If you are dealing with CAD imports into UE, you have already familiarized yourself with the Datasmith documentation and the required import steps. If you have not, it is important to note, that a direct import still has to be carefully prepared. As any program UE5 does exactly what the data tells it to do, so any asset visualized has to convey exact information into the engine.

If for example your normals are flipped in Rhino, they will be flipped in UE5 as well, so make sure to check your asset beforehand.

Best Practice to export CAD

In fact, this new approach in no way changes the basic requirement that the source must be perfect. Before implementing a CAD model make sure, that key features of your model are structured and applied correctly. Otherwise mistakes will happen and you will have to deal with hard to fix flaws.

To avoid problems, make sure to follow these steps correctly:

  • Normals must face the right way
  • A comprehensible layer structure
  • Add materials correctly with purpose
  • Group or connect objects as necessary
Prepare your CAD models before importing

Best practice to use CAD in UE5

After making sure the layer structure is comprehensible, place-holder materials are applied and the model is without any flaws, you can simply export the model through the rhino-datasmith plug-in. Afterwards you can then simply import and update the datasmith file through the UE-datasmith plug-in. Don't be surprised though, the model will be functional but probably not pretty.

CAD to UE 5 in progress

Shading issues and low quality materials will present themselves. Datasmith keeps the layer structure of CAD models, as well as materials. Since we applied named materials to all parts and layers we will have an easy time fixing any issues with the model.

A linear workflow in UE5 creates new tasks

Easy access, no Lowpoly but...

Thanks to the Internet, anything we advertise nowadays must be multimedia. Therefore our model won't simply stay inside of Unreal and be content with a simple render. However we cannot forget, that the model used, is still an imported CAD model and thus still has a high poly-count.

After production rework again

Now there is just the question about what to do next? How do we get the data out of UE 5 again and make it processable?

It is important access and delve deeply into the CAD production models. The correct export influences the entire downstream production. During the whole process we have kept the highpoly workflow, so the situation outside UE has hardly changed. Implementing the model has certainly become easier, but you also lose touch with the actual structure and data size.

This might answer the question "Is UE5 a gamechanger for CAD in realtime?". We think for a special workflow yes definitely!

At weltfern we tackle any challenge directly to be able to act efficiently in the workflows. If you have questions or need support get in touch.

Sources

Unreal Engine 5

https://www.unrealengine.com/en-US/unreal-engine-5

Images

All images and screenshots were taken for research purposes only, neither them nor the projects are for comercial use.

25. May 2022

Augmented Reality from your browser

Since the beginning of the collaboration with Adidas and the ever-advancing development of web browsers, weltfern has collaborated with major manufacturers like google, Mozilla, Snap.inc and others to push Augmented Reality features from your browser.

Sure, apps are handy, but we're talking about the little computers that everyone always has with them stuffed with photos and other apps. In fact Smartphone users face limited phone memory or choppy download speeds. WebAR is as simple as tap and go, this way brands have more options for introducing Augmented Reality experiences. As long as a device supports ARKit (Apple devices) or ARCore (Android devices) the basic requirements are there.

Here is a list of the current possibilities to access augmented reality from the browser. Yes, you read correctly ONLY and directly from your browser, even / especially on mobile! As a bonus, we have also listed some of the currently available frameworks.

World Tracking in WebAR (simple but classic):

Open a link URL and place an object in 3D on any surface near you. This option can be pointed out with an animation in the display after activating the camera (proven successful from a UX perspective). The most convenient AR use for events. Users and visitors scan QR codes with their smartphones and are directly in AR with the product.

WebAR with marker tracking:

This feature allows brands to set a specific 2D image (e.g. signs, logos, posters) as a special marker. This marker triggers the AR experience when detected by a phone's camera and adds another layer of immersion by allowing users to start the experience themselves (e.g. a barcode). This content extension is great for retail/packaging marketing.

WebAR with image tracking:

This feature allows brands to set a specific 2D image (e.g. signs, logos, posters) as a special marker. This marker triggers the AR experience when detected by a phone's camera and adds another layer of immersion by allowing users to start the experience themselves.

Location tracking WebAR:

This is effectively an extension that pulls content from a pre-determined location. For example, via a hotspot. Google Maps has proven that searching for a restaurant with an AR banner on the camera is fun. Of course, such a banner has to be created for this purpose.

Face tracking WebAR:

Face tracking opens the door for some crazy "filter" options. You know these as "Lenses" from Snapchat and AR effects from Instagram. Yet there are also more enhancement options that can be used like digital tattoos, makeup, and look changes. With a specific facial gesture, you can trigger the experience (e.g., eyebrows drawn together or "Say cheese").

This is ideal for brands looking to create a more personalized experience, and is a great source of shareable content. For online shopping of jewelry, glasses, hats, beauty products, etc.

Body tracking WebAR:

The camera can track specific body parts (or the whole body) and attach 3D models. Fitting virtual shoes or trying on virtual clothes are the best examples. This AR technology is becoming increasingly popular for metaverse experiments. Large companies are using their own solutions for this and AR apps like Instagram and Snapchat have been integrating this for years.

Excluded (in development), but soon to be relevant

Through the individual development of AR for market leaders and especially through organizations like Khronos, new experimental AR features based on the web browser keep coming. Google and also other browser producers are pushing this extremely. So you can always look forward to new options to expand your webstores!

WebAR Frameworks that can be used

AR.jsAR.js is based on the open source tracking library ARToolKit, JSARToolKit uses WebGL and Three.js to render 3D models over real objects. This framework enables open source AR, free, cross-platform and without installation on any modern device.
argon.jsThe argon.js framework provides a set of solutions and utilities to add an AR view to your web application in a platform- and technology-independent way. Whether you want to create a new AR application or add an AR view to an existing web application.
Awe.jsProvides a few other types of AR experiences such as augmented reality markers, location-based, and leap motion sensor AR. It uses WebRTC, WebGL, and get User Media device API to produce an AR experiences in the browser.
Three.ar.jsPretty much all web 3D applications are based on three.js! The AR tools here are a three.js helper library for creating AR web experiences. The tools run on WebARonARKit and WebARonARCore
Note: these are experimental apps for iOS and Android, for developers who can create augmented reality (AR) experiences using web technologies.
CodelabGoogle has compiled several examples for the creation of an AR web application in their "Codelab". This compilation offers a perfect start for developers.
This list shows some of the currently available frameworks, of course there are many more…

Start with WebAR for your Business

The beauty of WebAR is the fact that we are talking about an affordable option for your company here and now! If your product is hard to explain - this is your opportunity. Augmented Reality right from your browser is ideal for an entry into the future of marketing!

A good advice when starting to develop Augmented Reality applications is to try and experience the technical possibilities yourself. Of course you can contact us at any time, we will be happy to advise you!

Follow us

Follow us

Follow us

Follow us

TERMS AND CONDITIONSPRIVACY STATEMENT | DATENSCHUTZERKLÄRUNG
COPYRIGHT © 2022 weltfern ALLE RECHTE VORBEHALTEN.
TERMS AND CONDITIONSPRIVACY STATEMENT | DATENSCHUTZERKLÄRUNG
COPYRIGHT © 2020 weltfern ALLE RECHTE VORBEHALTEN.
TERMS AND CONDITIONSPRIVACY STATEMENT | DATENSCHUTZERKLÄRUNG
COPYRIGHT © 2020 weltfern ALLE RECHTE VORBEHALTEN.
TERMS AND CONDITIONS
PRIVACY STATEMENT
| DATENSCHUTZERKLÄRUNG
COPYRIGHT © 2020 weltfern ALLE RECHTE VORBEHALTEN.
TERMS AND CONDITIONS | PRIVACY STATEMENT | DATENSCHUTZERKLÄRUNG COPYRIGHT © 2020 weltfern ALLE RECHTE VORBEHALTEN.