How to Add Audio to a Babylon.JS Scene – Tutorial
This is the 3rd tutorial in our Bablylon.JS series.
In the first two, we created a scene, added the WebVR capability, added a skybox, and finally the ground.
But what is an environment without music and sound effects?
To begin, here is our javascript from the first two tutorials:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Tutorial 3 - Audio in Babylon js</title>
<!-- Babylon.js -->
<script src="https://preview.babylonjs.com/gltf_validator.js"></script>
<script src="https://preview.babylonjs.com/babylon.js"></script>
<script src="https://preview.babylonjs.com/loaders/babylonjs.loaders.js"></script>
<style>
html, body {overflow: hidden; width: 100%;
height: 100%; margin: 0; padding: 0; }
#renderCanvas { width: 100%; height: 100%;
touch-action: none; }
</style>
</head>
<body>
<canvas id="renderCanvas"></canvas>
<script>
var canvas = document.getElementById("renderCanvas");
var createScene = function() {
// Create scene
var scene = new BABYLON.Scene(engine);
// Default Environment
var environment = scene.createDefaultEnvironment({
createSkybox: false,
createGround: true,
groundSize: 1000,
groundColor: BABYLON.Color3.Green(),
enableGroundShadow: true,
groundYBias: 1
});
var skyMaterial = new BABYLON.StandardMaterial("skybox", scene);
skyMaterial.backFaceCulling = false;
skyMaterial.reflectionTexture = new BABYLON.CubeTexture("textures/skybox", scene);
skyMaterial.reflectionTexture.coordinatesMode = BABYLON.Texture.SKYBOX_MODE;
skyMaterial.diffuseColor = new BABYLON.Color3(0, 0, 0);
skyMaterial.specularColor = new BABYLON.Color3(0, 0, 0);
var skybox = BABYLON.MeshBuilder.CreateBox("skyBox", {size:1000.0}, scene);
skybox.material = skyMaterial;
// Enable VR
var vrHelper = scene.createDefaultVRExperience({createDeviceOrientationCamera:false});
vrHelper.enableTeleportation({floorMeshes: [environment.ground]});
var building = BABYLON.SceneLoader.Append("./", "vrchapel.glb", scene, function (meshes) {
scene.createDefaultCameraOrLight(true, true, true);
});
return scene;
};
var engine = new BABYLON.Engine(canvas, true, { preserveDrawingBuffer: true, stencil: true });
var scene = createScene();
engine.runRenderLoop(function () {
if (scene) {
scene.render();
}
});
// Resize
window.addEventListener("resize", function () {
engine.resize();
});
</script>
</body>
</html>
According to the documentation, the expected file type is mp3, but I found that wav file types also work without a problem.
Make sure that the audio file is located in the same or a sub-folder of the html file that will be loading it to play.
Now to load the audio file. I placed the instructions before the WebVR is loaded, immediately after the SkyBox is loaded:
// A little mood music
var music = new BABYLON.Sound("music", "mnkchant.mp3", scene, soundReady, {loop: true });
function soundReady() {
music.play();
}
Let’s break the instruction down. We begin by creating a variable to associate with the sound file. It is loaded into memory using the new BABYLON.Sound command. The parameters for the command are:
- The name of the sound (I used the same name as the variable for convenience)
- The URL/location of the sound file
- The scene to load the sound into (i.e. scene)
- Function to call when the sound is loaded and ready to play
- Options such as looping, volume, isPaused, etc
Once the sound is loaded and ready to play, the function soundReady is called which will play the music.
I found that this is great to get the music playing by default. When it is loaded in your server (local or a webserver), you will see an icon displayed in the top left corner:
This is because modern browser automatically silence any audio that begins playing in the browser (and let’s be honest, this is a good thing).
Clicking on the audio will allow you to hear the audio that you loaded.
In a later tutorial we will add the controls for the audio portion of the environment.
How to Change the Skybox and Ground settings in Babylon.JS – Tutorial
In our last tutorial on Babylon.JS, we loaded a 3D model into a basic environment. But we had a problem with the skybox and ground settings (you can see the results here). Here was the final code from last time:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Sample 1 using Babylon js</title>
<!-- Babylon.js -->
<script src="https://preview.babylonjs.com/gltf_validator.js"></script>
<script src="https://preview.babylonjs.com/babylon.js"></script>
<script src="https://preview.babylonjs.com/loaders/babylonjs.loaders.js"></script>
<style>
html, body {
overflow: hidden;
width: 100%;
height: 100%;
margin: 0;
padding: 0;
}
#renderCanvas {width: 100%; height: 100%;
touch-action: none; }
</style>
</head>
<body>
<canvas id="renderCanvas"></canvas>
<script>
var canvas = document.getElementById("renderCanvas");
var createScene = function() {
// Create scene
var scene = new BABYLON.Scene(engine);
// Default Environment
var environment = scene.createDefaultEnvironment({ enableGroundShadow: true, groundYBias: 1 });
// Enable VR
var vrHelper = scene.createDefaultVRExperience({createDeviceOrientationCamera:false});
vrHelper.enableTeleportation({floorMeshes: [environment.ground]});
var building = BABYLON.SceneLoader.Append("./", "vrchapel.glb", scene, function (meshes) {
scene.createDefaultCameraOrLight(true, true, true);
});
return scene;
};
var engine = new BABYLON.Engine(canvas, true, { preserveDrawingBuffer: true, stencil: true });
var scene = createScene();
engine.runRenderLoop(function () {
if (scene) {
scene.render();
}
});
// Resize
window.addEventListener("resize", function () {
engine.resize();
});
</script>
</body>
</html>
Let’s get those environment options adjusted so that they are not so distracting.
All of the environmental options can be adjusted at the time of creating the scene.createDefaultEnvironment. The only section that we need to adjust is the var environment = scene.createDefaultEnvironment({
section in the code above.
First, by default, the skybox is on. To adjust the skybox, we can add a couple of environment options:
var environment = scene.createDefaultEnvironment({
createSkybox: true,
skyboxSize: 150,
skyboxColor: BABYLON.Color3.Teal(),
enableGroundShadow: true,
groundYBias: 1
});In this code snippet, I have added the
createSkybox: true,
skyboxSize: 150,
skyboxColor: BABYLON.Color3.Teal(),code. The createSkybox: true, is redundant, but is nice to have if you want to be obvious in your code. To remove the skybox, change the true to false.
The skybox is by default a size of 20. With the second option, I have increased it’s size to 150.
The last line allows us to change the skybox color to teal.
Note that capitalization does matter!
Let’s do a similar process for the ground:
createGround: true,
groundSize: 200,
groundColor: BABYLON.Color3.Green(),By default, the ground is also on with a default size of 15.
To remove the ground, you can change the true to false.
There is a lot more we can do with the skybox and ground, but this will get us started for the next phase of our development.
You can see the the current look of our environment here: http://vrchapel.me/Tutorial2-skyboxandground.html
Now that we have a basic skybox and ground, let’s improve the skybox to make it a bit more realistic. Basic skyboxes are composed of 6 images, one for each face of a cube. Babylon uses a naming convention of the name of the image then _ab, the a representing positive or negative (p or n), and the b representing the x, y, or z coordinate. Thus, you get file names like skybox_nx.jpg to for the negative X image. You can download these and other skybox images from the babylonjs github repository. They are located in /playground/textures folder. https://github.com/BabylonJS/Babylon.js. I placed the textures in a new folder named “textures” in the same folder as my index.html file.
Time to load our new skybox. First, change the createSkybox in environment to false and comment out the associated options for the skybox:
var environment = scene.createDefaultEnvironment({
createSkybox: false,
// skyboxSize: 150,
// skyboxColor: BABYLON.Color3.Teal(),Next, we will create a variable to hold the skybox images. This is done by creating a new babylon standard material of the type “skybox” for our scene. We will set back face culling to false and then load the textures. The rest of the associated code is setting the coordinates (so that the images are laid out correctly), and setting the diffuse and specular colors to black:
var skyMaterial = new BABYLON.StandardMaterial("skybox", scene);
skyMaterial.backFaceCulling = false;
skyMaterial.reflectionTexture = new BABYLON.CubeTexture("textures/skybox", scene);
skyMaterial.reflectionTexture.coordinatesMode = BABYLON.Texture.SKYBOX_MODE;
skyMaterial.diffuseColor = new BABYLON.Color3(0, 0, 0);
skyMaterial.specularColor = new BABYLON.Color3(0, 0, 0);Finally, we create the skybox with a create box command and set the material of the box to the skyMaterial variable that we just created.
var skybox = BABYLON.MeshBuilder.CreateBox("skyBox", {size:1000.0}, scene);
skybox.material = skyMaterial;And now you have a skybox! The only problem is that the camera starts on the outside of the skybox instead of inside, looking at our building. We will correct that in the next tutorial. For now, you can use your middle mouse button to zoom inside of the cube.
Here is the final version of this tutorial and it’s associated code: http://vrchapel.me/Tutorial2b-skyboxandground.html
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Sample using Babylon js</title>
<!-- Babylon.js -->
<script src="https://preview.babylonjs.com/gltf_validator.js"></script>
<script src="https://preview.babylonjs.com/babylon.js"></script>
<script src="https://preview.babylonjs.com/loaders/babylonjs.loaders.js"></script>
<style>
html, body { overflow: hidden; width: 100%; height: 100%; margin: 0; padding: 0; }
#renderCanvas { width: 100%; height: 100%; touch-action: none; }
</style>
</head>
<body>
<canvas id="renderCanvas"></canvas>
<script>
var canvas = document.getElementById("renderCanvas");
var createScene = function() {
// Create scene
var scene = new BABYLON.Scene(engine);
// Default Environment
var environment = scene.createDefaultEnvironment({
createSkybox: false,
// skyboxSize: 150,
// skyboxColor: BABYLON.Color3.Teal(),
createGround: true,
groundSize: 1000,
groundColor: BABYLON.Color3.Green(),
enableGroundShadow: true,
groundYBias: 1
});
var skyMaterial = new BABYLON.StandardMaterial("skybox", scene);
skyMaterial.backFaceCulling = false;
skyMaterial.reflectionTexture = new BABYLON.CubeTexture("textures/skybox", scene);
skyMaterial.reflectionTexture.coordinatesMode = BABYLON.Texture.SKYBOX_MODE;
skyMaterial.diffuseColor = new BABYLON.Color3(0, 0, 0);
skyMaterial.specularColor = new BABYLON.Color3(0, 0, 0);
var skybox = BABYLON.MeshBuilder.CreateBox("skyBox", {size:1000.0}, scene);
skybox.material = skyMaterial;
// Enable VR
var vrHelper = scene.createDefaultVRExperience({createDeviceOrientationCamera:false});
vrHelper.enableTeleportation({floorMeshes: [environment.ground]});
var building = BABYLON.SceneLoader.Append("./", "vrchapel.glb", scene, function (meshes) {
scene.createDefaultCameraOrLight(true, true, true);
});
return scene;
};
var engine = new BABYLON.Engine(canvas, true, { preserveDrawingBuffer: true, stencil: true });
var scene = createScene();
engine.runRenderLoop(function () {
if (scene) {
scene.render();
}
});
// Resize
window.addEventListener("resize", function () {
engine.resize();
});
</script>
</body>
</html>
How to Load a 3D Model for VR with Babylon.JS – Tutorial
I recently began a major project using Babylon.js The reason I selected Babylon.js was the support for browser based game environment (at 60fps), and web-based VR (WebVR).
The first thing I wanted to do was load a 3D model that had been created in Blender. Here is how.
In this tutorial I’m going to quickly cover setting up your development environment and then demonstrate how to load a 3D model that was created in Blender (v.2.8). Make sure you export the final model as a glTF. I prefer the glb format over glTF as it incorporates the entire model, animation, and materials in one file rather than multiple files.
Once you have your model, you will need to decide where you will be testing your script from: a local server or a web server.
For faster iteration, I suggest setting up your development system with a local server. This is now much easier than in the past thanks to Python’s server tools.
To setup a local server, follow the directions on the Mozilla.org site: https://developer.mozilla.org/en-US/docs/Learn/Common_questions/set_up_a_local_testing_server
(Note: I found one error in the Windows directions: use ‘python -m http.server’ to launch your server instead of ‘python3’).
Of course, once you have a model loading, you will want to be able to share it with the rest of the world. I used AWS lightsail and launched a node.js instance to host my files.
Using Filezilla, I was able to quickly upload the html and glb files to the server and share them with the world. The files are stored in the /opt/bitnami/apache2/htdocs folder
Babylon.js can be completely encapsulated within the html document. For this tutorial, we are going to keep it as simple as possible and keep the html and script in the index.html file, only loading the glb file.
The html code is straight forward, loading the babylon framework and the gltf loaders. After the scripts are loaded, we can set the style for the document and the render canvas:
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Sample using Babylon js</title>
<script src="https://preview.babylonjs.com/gltf_validator.js"></script>
<script src="https://preview.babylonjs.com/babylon.js"></script>
<script src="https://preview.babylonjs.com/loaders/babylonjs.loaders.js"></script>
<style>
html, body {
overflow: hidden;
width: 100%;
height: 100%;
margin: 0;
padding: 0;
}
#renderCanvas {
width: 100%;
height: 100%;
touch-action: none;
}
</style>
</head>Now that we have the header taken care of, we can jump into the body. First we need to create a canvas and then create a function that will store the scene. Once the function is declared, we can setup the scene:
<body>
<canvas id="renderCanvas"></canvas>
<script>
var canvas = document.getElementById("renderCanvas");
var createScene = function() {
// Create scene
var scene = new BABYLON.Scene(engine);Next we will setup the default environment, enabling shadows on the ground. We will also turn on the VR option, so that anyone who has VR googles can see the object in VR.
// Default Environment
var environment = scene.createDefaultEnvironment({ enableGroundShadow: true, groundYBias: 1 });
// Enable VR
var vrHelper = scene.createDefaultVRExperience({createDeviceOrientationCamera:false});
vrHelper.enableTeleportation({floorMeshes: [environment.ground]});Now let’s get to the fun part; loading the model!
We are going to use the BABYLON.SceneLoader.Append command to make this happen. This allows us to load add the model to an existing scene. We will also set the camera and lights so that we can see everything.
var building = BABYLON.SceneLoader.Append("./", "VRChapel.glb", scene, function (meshes) {
scene.createDefaultCameraOrLight(true, true, true);
}); There is a lot happening with the Append command, so let’s break it down:
The 1st parameter – “./” – provides the location or folder of the file to be loaded. In this case the glb file is in the same folder.
The 2nd parameter “VRChapel.glb” the model to be loaded. If you saved your file as a glTF, only the model will be loaded, no textures, materials, or animations.
The 3rd parameter tells the engine where to load the model too, in this case the scene.
The 4th parameter is a function to create the camera and lights for the scene.
Finally, we return the scene and launch the engine. Notice the at the scene is describe before the engine is launched.
return scene;
};
var engine = new BABYLON.Engine(canvas, true, { preserveDrawingBuffer: true, stencil: true });
var scene = createScene();
Finally, we add a rendering loop, handle window resizing, and close the html page:
engine.runRenderLoop(function () {
if (scene) {
scene.render();
}
});
// Resize
window.addEventListener("resize", function () {
engine.resize();
});
</script>
</body>
</html>And there you have the minimum code to load a 3D model in Babylon.JS.
Of course, there is SO much more that we can (and will) do with this environment. This is just the starting point.
Special thanks to Adrianna Mott for the model! You can see the results of this tutorial online at: http://vrchapel.me/Tutorial1-loadModel.html
The only problem at this point is the skybox which is a little small and located inside the building. We will address that in the next tutorial.
How to Fill a Picker Wheel from a SQLite Database in Corona SDK – Tutorial
Last semester I had a group of students using Corona SDK who were using a database to list local establishments. They needed a way to easily display the list of locations in the SQLite database to the application.
While I had discussed the possibility of reading from a database and loading a picker-wheel in the textbooks Learning Mobile Application & Game Development and Beginning Mobile App Development with Corona, these students were relatively new to app development and needed a little more guidance on the process.
So, without further background, here is how to load a picker-wheel from a local SQLite database in Corona SDK.
To get started we will need to load the required plugins for sqlite and the picker-wheel widget tool in Corona. We will also make sure that the database is in the documents folder of the mobile device and not the resource folder.
*Note: I have created a database that lists all cities in the USA by zip code. I will be using it for this demonstration.
require("sqlite3")
local widget = require "widget"
-- Does the database exist in the documents directory (allows updating and persistence)
local path = system.pathForFile("zip.sqlite", system.DocumentsDirectory )
file = io.open( path, "r" )
if( file == nil )then
-- Doesn't Already Exist, So Copy it In From Resource Directory
pathSource = system.pathForFile( "zip.sqlite", system.ResourceDirectory )
fileSource = io.open( pathSource, "rb" )
contentsSource = fileSource:read( "*a" )
--Write Destination File in Documents Directory
pathDest = system.pathForFile( "zip.sqlite", system.DocumentsDirectory )
fileDest = io.open( pathDest, "wb" )
fileDest:write( contentsSource )
-- Done
io.close( fileSource )
io.close( fileDest )
end
-- One Way or Another The Database File Exists Now -- So Open Database Connection
db = sqlite3.open( path )
-- handle the applicationExit event to close the db
local function onSystemEvent( event )
if( event.type == "applicationExit") then
db:close()
end
endA quick note on the copy method. When you are working on a Windows system for development, you MUST use the “rb” (read binary) and “wb” (write binary) methods for reading and writing a database. This forces Corona to copy the file in the binary format. On a Mac, you can do a regular “r” and “w”. For greatest compatibility, we recommend always using rb and wb to avoid premature greying and extensive head-smashing-into-desk.
Time to load the data from the database. To begin, create a table to store the data loaded from the database (click here to see my tutorial on tables).
Next, create the SQL statement to load records from the database. I have limited the SQL statement to the first 20 records for simplicity (there are 1000’s of records).
After reading each row of data and creating a variable to store the city & state captured from the database, we add the text to the data table that will be used by our picker-wheel.
local data = {}
local sql = "SELECT * FROM zipcode LIMIT 20"
for row in db:nrows(sql) do
local text = row.city..", "..row.state
table.insert(data, text)
end
Time to create the picker-wheel. We are going to keep it simple; just one column of data that contains the city and state information.
The picker-wheel will be 258 pixels from the top of the screen (for no real reason, just looked good on my test app), with a font size of 18 and a default color of black. The currently selected city & state will be red.
--Create storage for picker wheel
local columnData = { {align = "left", startIndex = 3, labels=data } }
-- create the picker widget with columnData
local picker = widget.newPickerWheel({
top=258,
fontSize = 18,
fontColor={0,0,0},
fontColorSelected = { 1, 0, 0 },
columnColor = { 0.8, 0.8, 0.8 },
columns = columnData,
})
Just three more things to do: add a button to return the selected data to the console, setup the function to print the selected value, and finally, add a event listener to close the database if the user abruptly closes the app.
local function showValues( event )
-- Retrieve the current values from the picker
local values = picker:getValues()
-- print the selected values
print( "Column Information: " .. values[1].value )
end
local getValuesButton = widget.newButton
{
left = 10,
top = 150,
width = 298,
height = 56,
id = "getValues",
label = "Values",
onRelease = showValues,
}
-- system listener for applicationExit
Runtime:addEventListener ("system", onSystemEvent)
And that is how you load a picker-wheel from a local SQLite database. This tutorial has been added to the Corona textbooks available on our website.
FYI, the students did a great job on their project!
If you would like the video tutorial of working with a Picker wheel and SQLite, checkout the link below!
Reflections on Personalized Education
Traditional education assumes a transmissive format of instruction. With #VR we can leave behind the traditional “sage-on-the-stage” and enable new #epistemology approaches, enabling learners to experience #education instead of trying to recall it from a lecture. When we begin to include #AI agents, #bigdata; when we are creating true #personalizededucation that adapts to the needs of the student, then we are accomplishing what generations of educators have sought to accomplish.
More to come, this is just the beginning!!
Certbot Not Renewing
First, this post is more for me as a reminder should I have this problem again. But, if it helps you resolve your problem, yea!
Problem (tl;dr version): Certbot didn’t update the ssl certificates on one of my servers, resulting in ugly messages about the websites not being safe.
Long Version: I’m hosting my own sites on Amazon EC2 servers running AMI flavor of Linux. About a year ago I upgraded from standard webpages to ssl to make Google and other search engines happy. Everything was going fine until recently when the site didn’t renew the certificates for SSL from Let’s Encrypt.
While I had followed the Amazon tutorial exactly (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/SSL-on-an-instance.html#letsencrypt) Certbot wasn’t running any longer. After troubleshooting, I found that it was a common problem with the latest versions of certbot. There were a lot of solutions provided by various people, none worked correctly. What did work is:
Solution:
On the github website (https://github.com/certbot/certbot/issues/1680#issuecomment-358728515) I found a solution that worked:
sudo rm -rf /opt/eff.org/*
pip install -U certbot
sudo certbot renew –-debug
I did have to make one change since I was using the -auto version of certbot:
sudo ./certbot-auto renew –-debug
Now the solutions is available should I (or anyone else) need it!
Lumberyard, Lua, Corona Projects Update
We have been working on serveral major projects at Burtons Media Group. We thought we would take a moment and provide an update on individual status of the projects:
Amazon Lumberyard Game Development Fundamentals: Space Explorer
The textbook is mostly complete with the exception of the final chapter where we cover visual scripting. We are semi-patiently awaiting the release of Lumberyard 1.10. This update promises to be a major one with the inclusion of Script Canvas, Amazon Lumberyard’s visual scripting tool set. Expect to see the release of this textbook and associated on-demand video course soon after the 1.10 release.
Corona Development On-Demand Course
The on-demand course is in active development with completely new and updated videos and content. While based upon our Learning Mobile Applicaition & Game Development and Beginning Mobile App Development textbooks, we are adding additional lessons to make this a great value above and beyond the textbooks. We will be providing a discount if you previously purchased on of the Corona Textbooks.
Work continues on the isometric game development book. We will be using the Million Tile Engine plugin to handle much of the heavy lifing and focus on the isometric game development process in this textbook.
Learning Lua
Lua seems to be at the center of every project that we develop. While it was not intentional, it has definitly become the “need to know” scripting language at BMG. To that end we have started developing a textbook and on-demand course content on Lua that will cover everything from the basics through using Lua for Machine Learning (Torch) and implementing Lua as a part of an engine.
Burton Institute of Technology
We at Burtons Media Group were recently asked when we would begin to provide on-demand training for Lumberyard, Corona, and Lua. In the past we have focused on digital-print (i.e. pdf and ePub) plus making games.
As we surveyed the landscape (and with Dr. Burton’s extensive knowledge of online learning) we have decided to begin actively developing online certificate courses. Production has already begun and you can expect to see BIT courses via the BurtonsMediaGroup.com website in the very near future.
Getting Started with Deep Learning
Why Deep Learning?
Several months ago one of my former students, Dr. Julia S. Slige, began posting a series of tutorials on using R for data mining. Julia even has a book on the topic published by O’Reilly. Her posts and subsequent research got me to thinking about Deep Learning, a key part of my ongoing research into adaptive/immersive learning using game engines.
As a result of these musings, I have begun the process of learning to use the latest Deep Learning tools so as to enter this next phase of research. As I like to share what I am learning, writing/blogging, as Bruffee (1999) postulates, helps to maintain the conversation that is learning. I will be recording my musings and what I have learned along the way. Who knows, it might even result in another textbook (that is how all of my previous textbooks got their start).
One of my goals as I share what I have learned is to make it easier for those who follow the path that I am blazing. This is still early days of AI, Deep Learning, Machine Learning and lots of similar buzz words. It can be confusing and overwhelming to make sense of it all so that you can apply the concepts to what you are interested. Hopefully, after reading these blogs, it will be a little less confusing.
What is Deep Learning
There is a great deal of hype and excitement around Artificial Intelligence, Machine Learning, Neural Networks, and Deep Learning. Unfortunately, the terms are frequently being used interchangeable which only adds to the confusion. Please understand that each description below is a broad generalization. Each of these areas is immense with many methods and theories. So before we go any further, let’s agree on the general terminology:
Artificial Intelligence
Artificial Intelligence or AI as it is commonly referred to, is the catch-all term for anything related to AI. You’ve seen the movies; robots and computers that are able to interact and learn from their environments to accomplish their mission, some bad (Terminator, HAL), some helpful (R2D2), some annoying (C3PO).
The goal AI is to create software that learns as it interacts with the environment. At this time, implementations of AI are very narrow and specialized for specific fields. AI includes the subdomains of Machine Learning, Neural Networks, and Deep Learning. While AI has been around for years, only with the recent availability of cloud-based computing are the resources available to begin to simulate AI.
Machine Learning
Machine Learning (ML) relies upon algorithms to parse data, learn from the data, then make a prediction based upon the data. ML systems are ‘trained’ using a lot of data (and I mean a LOT of data) to make accurate predictions. You are probably already using ML in your everyday life: Spam filtering! If you use a cloud-based email system such as Gmail, then you are using ML. Have you noticed that the first time you receive a message that is probably spam, it goes into your inbox, but after that, similar messages go straight to your spam folder. Gmail learns from you, based on your telling it what is spam and what isn’t.
Neural Networks
Neural Networks (NN) builds on ML. The concept is that just as the human brain uses neurons to pass information, neural networks can create pathways that show lead to the correct answer more quickly. ML makes a lot of mistakes as it is learning. Neural Networks learn more quickly because a weight or value is given to paths that result in the correct answer. These paths are stored in hidden layers.
Neural Networks are very processor intensive, even more so than ML. Only with the advent of parallel processing and GPUs have Neural Networks begun to accomplish what they are capable of doing.
Deep Learning
Deep Learning (DL) combines ML and NN to accomplish the next phase of AI. By using sets of algorithms that are then layered. Deep Learning is different from Neural Networks in that it uses more layers in the building of learning paths. Additionally, Neural Networks are usually supervised as they learn. Deep Learning can be supervised or unsupervised (or a combination) as it goes through it’s learning process. At this time Deep Learning is providing exciting advances in optical recognition i.e. read a stop sign, process natural language, speech systems, translations, even adding closed captioning to YouTube videos.
Why Focus on Deep Learning?
Why will I be skipping over implementations of Machine Learning and Neural Networks? It comes down to goals. While I will be sharing certain aspects of ML & Neural, the aspects of AI that I am interested in are concentrated in the Deep Learning domain.
My research is focused on creating immersive educational environments that individualizes instruction based on student performance. My wife, Rosemary, and I created one of the first fully online schools back in the 90’s. Our continued research and interest in game design, gamification, and now Deep Learning, are all focused on building a better, pedagogically superior, learning experience for all learners, regardless of location, background, disabilities, language, or previous experiences.
Programming Languages and Tools for Deep Learning
AWS/Google/Microsoft Azure/IBM Bluemix
All major cloud companies are trying to stake their claim in the AI field. Each has slightly different offerings and tools (and pricing for those tools).
I have received research funding in the form of free access to the cloud infrastructure (no real money actually received) for each of the four mentioned companies for cloud computing over the last 5 years.
I have settled on using Amazon AWS for my personal and research work. While the services, cost, and resources do not vary that much from company to company, Amazon’s customer-centric nature clicked with how I like to do things, thus that is where the majority of my time and energy has been spent. Also, AWS supports both Lua Torch and Python Theano, the two dominant tool sets that we will be (at least intially) exploring.
Lua
I have to admit, I was excited to see that there was a Lua tool set for Deep Learning. I have been developing with Lua for over 15 years and used it intensely for the last 7 years. If you haven’t seen my Lua tutorials on Youtube, they can be found here.
The dominant tool in Lua is Torch. It is considered a very flexible tool that keeps the easy of use of Lua.
Python Theano
Python seems to have broader support (and many tools) than Lua in deep learning.This might be caused by Python is in wider general use than Lua, which naturally makes it more attractive.
The dominate tool is Theano. While there are many others in development and gaining in popularity (most notably Cafe2), Theano remains the primary Python Deep Learning tool.
Other languages:
R and Julia are also being used for deep Learning but do not appear to have the depth of support Lua and Python enjoy at this time.
Deep Learning is still relatively young as far as tools. There are a lot of things in development, but from my research, The lead tools in Deep Learning are Lua Torch and Python Theano, so that is where I will be concentrating my efforts at this stage.
Next Steps
Now that we have an idea of what Deep Learning is and a few simple examples of what it is capable of doing, I will be exploring the tools. In my next blog post, we will take a closer look at Python Theano and Lua Torch.
While I lean toward Lua and Torch just because of long familiarity, I don’t want to make the mistake of passing on a better set of tools due to familiarity. Since this is a new skill set that I am learning, if I need to improve my Python skills to maximize success, I am very willing to do so.
Next: Lua Torch v. Python Theano
References:
Bruffee, K. A. (1999) Collaborative learning. John Hopkins University Press
Slige, J. S. & Robinson, D. (2017) Text mining with R: A tidy approach. O’Reilly Media
Slige, J.s. (2016) Data science-ish (blog)
Why switch to Amazon Lumberyard?
Recently I was asked why I had selected Amazon Lumberyard over Unity, Unreal or other 3D game engines.
There are several reasons I chose to go with Lumberyard.
Unity:
Yes, Unreal has been/is free for education. However, their royalty structure for a semi-successful game is a little frustrating and expensive. Again, I do not have a problem paying a little if my game is successful, but I generally avoid royalty structures (unless I’m the recipient :-).
Note: I expect a certain level of professionalism. I run a game design degree that is ranked in the top 20% internationally. I realize that I’m not at a big name university, but a cordial, polite response to my questions is expected no mater how big the organization.
Soon after that initial email, I was speaking with engineers, developers, and even the head of one of the divisions at Amazon. Amazon even sent one of their people to spend the day at the University I teach to give a series of tech talks and game dev talks to my students. When compared to how other companies treat education, this was the deciding factor. I love Amazon’s customer-centric focus.
Access to Engine code
For most indie developers this isn’t a big factor. But if you have a HUGE project that you plan to span the next 20 years of your life (yes, I have project that I am working on that meets this criteria) then the availability of the engine code could be critical. What if the company goes out of business? What if they go left when you need to go right? While I do not have any interest in being in the ‘making game engines’ business, it could be an issue long term.
Unity: Scripting in Unity can be done with C# (as I understand it, javascript is being depreciated?). There are a few other languages as well, but C# is the one that most people use.Unreal: Default scripting is C++. While this is great if your students are computer science majors, it’s a problem if you’re focused on game design.
Lumberyard: Default scripting is Lua. Yep, Lua, the same language that we use for mobile game development with Corona SDK. Learning Curve: 0
Be sure to check out our new books on learning Amazon Lumberyard to make this a little less painful!
This area is a toss up between the big three engines. Each implements VR slightly differently, but all have made it easy to do.
For the University game development program and my personal use, we are going 100% in on Lumberyard. The engine is better (IMHO) for game dev, the cost is perfect, and the company is large enough that I don’t have to worry about if it exists in the future. With the added bonus that I get the engine source code so that I can continue to tweak it if they do decide to stop development, I have a path forward.
Learning Lua Scripting- Part 5 – Math and Math Library
In this Learning Lua Scripting tutorial we will examine how to use the Lua Math Library.
Lua has the basic math operations that you would expect to find in a modern scripting language.
^ – exponential
* – multiplication
/ – division
% – modulus or modulo
+ – addition
– – subtraction or unary
As of version 5.3 of Lua, numbers are stored internally as either integers or double (64 bit) by default.
Prior to 5.3, all numbers were stored as doubles.
Lua follows the standard order of precedence for operations: ^, not, #, unary, *, /, %, +, –
Library:
The math library can be accessed with the math keyword without special loading:
[codebox 1]
Results in 3.1415926535…
The math library has all the functions that you would expect including sin, cos, random, randomseed, etc.
If you wish to generate a random number, be sure to randomize the number generator:
[codebox 2]
Resources
If you would like print resources, there are several books on Lua available.
Programming in Lua by Roberto Ierusalimschy, one of the lead architects of Lua. Great technical intro to the language
Lua Programming Gems by L. de Figueiredo, W. Celes, and R. Ierusalimschy is an older (2008) collection of code snippets that can be useful.
Editor –
We used the Zerobrane editor in all of our video demonstrations.
Our books:
We have several books on Corona and Amazon Lumberyard (both of which use Lua as their scripting language):
Learning Mobile Application & Game Development with Corona – Learn to program in Lua and how to make mobile apps! eTextbook for those who are new to programming.
Beginning Mobile App Development with Corona – Introduces mobile application development for those who already know how to program.
Game Design Fundamentals with Amazon Lumberyard – For those who are new 3D Game development, this eTextbook introduces how to make a game using Blender, GIMP, and Amazon Lumberyard.
The idea of writing a textbook on the Lua Scripting language has been floated to me. While I greatly value Dr. Ierusalimschy, our styles of instruction are very different. Leave me a comment if you would like to see a Lua Scripting textbook.