What are Augmented Images and How to add them to Processing Android?
After in-depth study of Ar Core Augmented Images, I came to the understanding that it’s a really convenient and effective functionality that can used for a plethora of tasks.
With Augmented images being introduced in Google I/O ’19 it has been around for some time now. To give you an idea what Augmented Images are and how it works, allow me to give you a demonstration. Scan the image given below with any Barcode scanner app (or Google Lens) and then use the newly opened webpage to scan the Hiro-Image.
What we did here was to store the Hiro Logo as an Augmented Image and link it to a 3D model. The website that opens up when the barcode is scanned is a simple web application that stores the model, image and the code to link and process them.
Now, that you have a basic idea about what Augmented Images are, lets move to the next step and see how we can make Apps that are able to use this functionality.
Google has provided detailed documentation regarding the Augmented Images and how to implement them. There are also codelabs where you can make a simple app on your own. Apart from that here are some blogs that I personally found extremely helpful to learn more about AR Image Augmentation:
After reading these blogs you will be able to create your own AR apps that can use Augmented Images. (Google has opensourced Sceneform and now most of the developers are using SceneView, which is an unofficial alternative to the Sceneform.)
Now, that we have a basic knowledge of what Augmented Images are and how we can develop apps using that, the main question arises. How to implement it in Processing Android. Though, adding the Augmented Image Functionality is a relatively simple task, the main issue arises with adding the 3D-models.
ARCore can track up to 20 images simultaneously. ARCore will not simultaneously detect or track multiple instances of the same image.
Each Augmented Image database can store information up to 1,000 reference images. There’s no limit to the number of databases, but only one database can be active at any given time.
Images can be added to an Augmented Image database at runtime, up to the 1,000-image per-database limit. It’s possible to download previously created databases using a network connection.
(TLDR: We can store as many 3D models as we like, but only 1000 images(that are scanned like the Hiro logo above) for each 3D model.)
Due to this, there are multiple ways using which we can add AR Augmented Image to processing.
- Add all the models and images before hand and not allow the user to add any more images/models.
- Add all the models before hand but allow the users ability to add more images to identify the existing models.
- Create empty database and let the user add models and images to their liking.
The first method is widely used by apps such as Furlenco(though, it uses markerless AR) where the developers have a fixed database of models( and images ) and don’t need any input from the user.
Please note, that here by user I mean the App-Developer. As all the models are predefined in an app and no application allows the end user to add their own 3D models.
The second method might come in handy we have a fixed set of models and want the user to play around with different images and add them to the database(and link them with a particular model). Eg: An app that teaches childern Alphabets. Here, the user would be happy to add their own variations of letters to the existing Alphabet models.
The third variation is what I personally suggest to be added to Processing. After much brain storming, I came to the conclusion that the developers will use Processing IDE to make their own AR apps that can recognise various AR models. And since, it’s next to impossible for us to maintain a database(of 3D models) big enough to satisfy all users, why not we allow the developers to add thier own 3D models to their apps and store them either locally or on cloud storage(like Firebase).
This way, we not only allow the developer to run wild with their creativity but also facilitate the creation of a custom app, as per the user’s discretion.
So, I propose that we incorporate the functionality Augmented Image using AR Core and create an empty database, such that the developer can store their models and images locally/on-cloud and thus create a new generalised functionality for Processing Android-AR.