How to control a robot with OpenSpace3D

In this tutorial you will learn how you can build an interactive Augmented Reality (AR) app using OpenSpace3D. With their latest release OpenSpace3D included the ARToolKit5 library.

Components for this tutorial

• OpenSpace3D (on Windows)
  • I used the portable version which is available here.
• These images which will be used as NFT markers

• And as usual for an Augmented Reality app you need a camera

Setup the model

We are going to start OpenSpace3D and load the supplied robot model.

1. Start OpenSpace3D
2. Add the robot model (Andy) to the scene.
   • Right click on the scene root
   • Add mesh
   • Select next
   • Select the robot mesh: “OS3D\_root”\OpenSpace3D\assets\models\common\andy\mesh\andyv2_andy.mesh
   • Select next
   • Select *Add script* and select “OS3D\_root”\OpenSpace3D\assets\models\common\andy\material\andyv2.material -> Open
   • Import

Now you have Andy in. If you select it in the Scene tree you get a little ‘+’ icon next to it which let’s you explore the model and all it’s animations.

Activate the material

Good, now we have the basic scene. But, our model is not looking very good. We need to enable the material to make it look good.

1. Right click the Scene root
2. Set material scheme
3. In the dialog select basic_mat

Now Andy should appear with a white and blue material with the OpenSpace3D logo on his chest.

Light and shadow

The scene is still a bit dark and also it looks much more realistic if we enable shadows. So let’s just do that.

1. Right click on ARAndy_dummy
2. Add light -> Ok
3. In the dialog ensure you select Cast shadow
4. Also make your light a direct light.

Now we add a plane to which the shadow is projected onto and enable the shadow mode.

1. Right click on ARAndy_dummy
2. Add Mesh -> Next
3. Select “OS3D_root”\OpenSpace3D\assets\primitives\prim_plane.mash -> Next -> Import
4. Right click on Scene root / Add resources to scene
5. Select “OS3D_root”\OpenSpace3D\assets\materials\transshadow.material -> Open
6. Right click the plane you just added -> Edit materials
7. For material name select ASSET/TransShadow
8. Right click on Scene root -> Set shadow method
9. For Preset method select presetPSSM

Augment it

If we run the scene now you can see that Andy is doing some work and that it has a shadow.

Add the AR camera

Now we want Andy to be augmented onto the robot image.

1. At the bottom inside the groups tab double click Scene
2. In the scene tab right click/input/AR capture -> OK

Add the marker

If you play the scene now you see that Andy is rendered in front of the camera stream.

Now we need to place Andy on a marker:

1. Right click in the scene tab at the bottom
2. Input / Add marker
3. For Bitmap path select the image of the robot. This starts the marker training which may take a bit.
4. Marker size needs to be changed to 1 or bigger otherwise Andy will be really large. Play around with this setting till Andy appears to your liking. (I set it to 2)
5. In Object name select the ARAndy-dummy node.

Ok, now Andy is placed on top of the marker when the marker is shown. However, it is also rendered when the marker is not shown which is kind of disturbing. So let’s fix that.

1. Right click in the scene tab at the bottom
2. Select object/hide
3. Set Source name to ARAndy_dummy
4. Select Hide on start

Now Andy is no longer rendered when we play the scene. To make Andy appear on the marker do the following:

1. Right click AR marker inst
2. Select found: Now a line is shown starting at AR marker inst and following your mouse pointer
3. Right click on hide inst and select show

This makes Andy appear as soon as the marker is detected. Do the same for Lost to hide Andy when the marker is not visible any more.

A video, demonstrating the above steps can be viewed here.

Control Andy

Now that you can show and hide Andy based on the robot marker the next step is to show him what to do based on the other markers.

1. Add three more markers for girlbot, beerbrake and computer.
2. Right click in the scene tab at the bottom
3. Select object/animation switcher
4. Add the animation you would like to be played for each of the marker (robogirl, beerbrake, computer) to the Animations list.
5. Right click on one of the new markers. Select found, right click on the animation switcher and select the animation you would like to play when the selected marker is found. Be aware that one animation is set to default so you won’t see the name of this animation in the list.
6. Repeat this step for all of your markers
7. Don’t forget to select some animation (or non) when the marker isn’t visible any more (lost)

You can see my final app here:

Conclusion

You have learned how you can setup an augmented reality app using OpenSpace 3D. Additionally this tutorial gives you some background how you can use markers to control 3D objects like our Andy and tell a nice story with it.
Try to think this example further and make real objects react on images shown to them (or to a camera). Maybe using a Raspberry PI or an Arduino.

I’d love to hear from you and see your stories and apps. Let’s keep in contact on Twitter (https://twitter.com/artoolkit_) or @Thor_Bux. Also for questions and feedback please leave a comment below.