Machine Learning Archives - RIDE https://ride.ict.usc.edu/tag/machine-learning/ Rapid Integration & Development Environment Thu, 08 Sep 2022 18:21:39 +0000 en-US hourly 1 https://wordpress.org/?v=6.8.3 https://ride.ict.usc.edu/wp-content/uploads/2020/05/cropped-RIDE_Logo_Icon_250x-32x32.png Machine Learning Archives - RIDE https://ride.ict.usc.edu/tag/machine-learning/ 32 32 Barracuda API https://ride.ict.usc.edu/support/documentation/examples/barracuda-api/?utm_source=rss&utm_medium=rss&utm_campaign=barracuda-api Fri, 25 Mar 2022 22:26:34 +0000 https://ride.ict.usc.edu/?page_id=4430 This example demonstrates how to examine and run unit tests on various Machine Learning models residing within/outside RIDE backed with Unity Barracuda framework.

The post Barracuda API appeared first on RIDE.

]]>
Machine Learning

Barracuda API

Overview

This example demonstrates how to examine and run unit tests on various Machine Learning models residing within/outside RIDE backed with Unity Barracuda framework

Page Contents

Barracuda-API-1
ExampleBarracudaAPI

Purpose

This example demonstrates how to examine and run unit tests on various Machine Learning models residing within/outside RIDE backed with Unity Barracuda framework. 

How to Use

Run-time, either inside Unity editor or standalone:

  1. Select the Inference Type from the dropdown list. For this example here choose Classification.
  2. Load the Machine Learning model (as ONNX) into the scene by pressing the Load ONNX Model button. This model can lie within the Unity project or anywhere locally. For this example, in the file manager that pops up, browse to Assets/Ride/ML/BarracudaAPI/Models/Classification and select the classification_efficientnet-lite4-11.onnx file there and click Open.

  3. Load the parameters defining the sizes of various I/O for this model. Click on the Load Parameters button. This file can lie within the Unity project or anywhere locally. For this example, in the file manager that pops up, browse to Assets/Ride/ML/BarracudaAPI/Models/Classification and select the parameters.txt file there and click Open.
  4. Load the labels defining the various classes for the classification. Click on the Load Labels button. This file can lie within the Unity project or anywhere locally. For this example, in the file manager that pops up, browse to Assets/Ride/ML/BarracudaAPI/Models/Classification and select the labels_map.txt file there and click Open.
  5. Select the Input Type from the drop down list. Choose Single Image.
  6. Load in the image input by pressing the Load Input button. This image can lie within the Unity project or anywhere locally. For this example, in the file manager that pops up, browse to Assets/Ride/ML/BarracudaAPI/Models/Classification/Test data and select the water_bottle.png file there and click Open.
  7. Click on the Run Inference button to the result of this Classification model.


Other Use Cases

Classification from live webcam feed:

RIDE MSELI:

 

Scene Location & Name

Assets/Ride/ML/BarracudaAPI/Scenes/ExampleBarracudaAPI.unity

Setup Requirements 

For the Ride MSELI example, make sure to unzip Assets/Ride/ML/BarracudaAPI/Models/MSELI_LSTM/Test data.zip before running the experience.

Note

Currently this API does not work in WebGL mode. Investigating compatibility in a future update.

The post Barracuda API appeared first on RIDE.

]]> Speech Recognition https://ride.ict.usc.edu/support/documentation/examples/speech-recognition/?utm_source=rss&utm_medium=rss&utm_campaign=speech-recognition Tue, 06 Apr 2021 00:43:33 +0000 https://ride.ict.usc.edu/?page_id=3519 Demonstrates how to use Microsoft speech recognition in RIDE.

The post Speech Recognition appeared first on RIDE.

]]>
Machine Learning

Speech Recognition

Overview

Demonstrates how to use Microsoft speech recognition in RIDE.

Page Contents

ExampleSpeechRecognition
ExampleSpeechRecognition

Purpose

Demonstrates how to use Microsoft speech recognition in RIDE.

How to Use

  1. Hit Play
  2. Click on the Continuous speech button
  3. Talk into your microphone
  4. See that the words that you spoke appear on the UI

Scene Location & Name

Assets/Ride/Examples/Speech/SpeechRecognizer/ExampleSpeechRecognition.unity

Setup Requirements 

A microphone is required for this scene to function properly.
 
Add the Assets/Ride/Examples/Speech/SpeechRecognizer/SpeechManager.prefab to enable the service and Assets/Ride/Examples/Speech/SpeechRecognizer/SpeechRecognitionMenuobject.prefab into the scene in order to provide an interface for the user.
 
Reference ExampleSpeechRecognizer.cs to see how the speech recognizer service is setup.

The post Speech Recognition appeared first on RIDE.

]]> Localization https://ride.ict.usc.edu/support/documentation/examples/localization/?utm_source=rss&utm_medium=rss&utm_campaign=localization Wed, 27 Jan 2021 20:06:27 +0000 https://ride.ict.usc.edu/?page_id=2452 Demonstrates how to load, display, and switch between different languages

The post Localization appeared first on RIDE.

]]>
Machine Learning

Localization

Overview

Demonstrate how to load, display, and switch between different languages.

Page Contents

ExampleLocalization

Purpose

The example demonstrates how you can load, display, and switch between different text strings at runtime.  This feature allows you to display your UI/UX in different languages, catering to the end users’ preferred language.

How to Use

The IGroupSystem allows you to logically group agents together. Agents can exists in one or more groups.  Groups can have one or more sub groups.  You can also build relationships between groups.

Scene Location & Name

Assets/Ride/Examples/Services/ExampleLocalization.unity

Setup Requirements 

This scene uses the “Localizer” object which allows you to load different language text files.  Examine the files “en-us.txt” and “es-sp.txt” to see how you can add more strings to those files, or use them as examples for creating new localization .txt files in the language of your choosing. When you are ready to load them at runtime, drop them to the  end of the languages array on the Localizer object. 

In the scene, under Canvas is a game object called “Dropdown”.  Select it and, in the options section, type the name of your language that will display. 

ExampleLocalization Script

Add this script to an object in your scene to enable the switch of languages at runtime.

ILocalizationSystem Script

Referenced by ExampleLocalization script for group definitions. API reference here.

The post Localization appeared first on RIDE.

]]> Training TakeCover ML https://ride.ict.usc.edu/support/documentation/examples/training-takecover-ml/?utm_source=rss&utm_medium=rss&utm_campaign=training-takecover-ml Fri, 13 Nov 2020 20:28:16 +0000 https://ride.ict.usc.edu/?page_id=1722 Rapid training of mlAgent units for modelling cover behaviour by inference. This scene links to an external python application over Unity TCP containing user-defined observations created in an academy for which the unit “brains” undergo training.

The post Training TakeCover ML appeared first on RIDE.

]]>
Machine Learning

Training TakeCover ML

Overview

Rapid training of mlAgent units for modelling cover behaviour by inference. This scene links to an external python application over Unity TCP containing user-defined observations created in an academy for which the unit “brains” undergo training.

Page Contents

ExampleTrainingTakeCoverML
ExampleTrainingTakeCoverML

Purpose

Rapid training of mlAgent units for modelling cover behaviour by inference. This scene links to an external python application over Unity TCP containing user-defined observations created in an academy for which the unit “brains” undergo training.

How to Use

Prerequisites

Setup the python environment following these steps:

https://github.com/Unity-Technologies/ml-agents/blob/release_12_docs/docs/Installation.md

Approximate Procedure:

  • Python 3.7 64bit or higher
  • Python and Scripts folder needs to be in PATH, eg:
    • C:\Python37_64
    • C:\Python37_64\Scripts
  • If you are using Powershell, make sure you enable scripts. From admin Powershell, type:
    • Set-ExecutionPolicy unrestricted
  • Install pip
  • Install torch:
  • Install ml-agents
    • pip3 install mlagents

Steps

In order to utilize this example, you’ll first need to acquire the following Unity ML-Agents Toolkit core components: ml-agents-0.15.1, ml-agents-env, and ml-agents-release_12. Next, add the ML Agents v1.0.7 python training environment, which can be obtained through the Unity Package Manager.

Make Sure RIDE_ML_AGENTS is defined in the project scripting defines.

After completion, open a command window prompt at: 

/ml-agents/ml-agents-env

Enter the commands:

  1. ml-agents-env-release12/Scripts/Activate
  2. cd ../ml-agents-release_12

To start the training programming, enter the following command:

  1. mlagents-learn config/ppo/basic.yaml –run-id=<run-identifier> –train –time-scale=100
    1. Note, if you a using a previous run-identifier than pass –resume as well

Then hit play in the Unity scene ExampleTrainingTakeCover scene and you will see the agents begin to train.

To watch how effective the training is, open a new command prompt in /ml-agents/ml-agents-env and enter the following:

  1. ml-agents-env-release12/Scripts/Activate
  2. cd ../ml-agents-release_12
  3. tensorboard –logdir results –port 6006

Scene Location & Name

Assets/Ride/Examples/Behaviours/MLBehaviours/ExampleTrainingTakeCoverML.unity

Setup Requirements 

Utilize mlAgent for your scene with the ExampleTrainingTakeCoverML script and TrainingArea_TakeCoverML prefab that contains the TrainingArea_TakeCover script, floor/walls, cover, goals, Agent (script), Enemy, and mlAgent objects.

ExampleTrainingTakeCoverML Script

First, add ExampleTrainingTakeCoverML script to an object in your scene.

TrainingArea_TakeCoverML Prefab and TrainingArea_TakeCover Script

Next, import the TrainingArea_TakeCoverML prefab into your scene to train agents for cover behavior in conjunction with an external python application.  

This prefab that contains the TrainingArea_TakeCover script, floor/walls, cover, goals, Agent (script), Enemy, and mlAgent objects.

Customizing and Extending Scene  

Create a prefab variant of TrainingArea_TakeCover to begin constructing your own training environments. 

Recommend building a comparison scene to evaluate two approaches to your problem set. See Behaviour Comparison as an example for how to structure a side-by-side comparison of agent behaviour.

Unity Machine Learning Agents Information

Unity ML-Agents Toolkit

ML Agents Package Manual

Unity ML-Agents Tools

Unity ML-Agents How-To

Unity ML-Agents Intro – Blog

The post Training TakeCover ML appeared first on RIDE.

]]> Behaviour Comparison https://ride.ict.usc.edu/support/documentation/examples/behaviour-comparison/?utm_source=rss&utm_medium=rss&utm_campaign=behaviour-comparison Fri, 13 Nov 2020 20:14:54 +0000 https://ride.ict.usc.edu/?page_id=1715 This scene demonstrates how one might analyze side-by-side the performance of two approaches for agent cover behaviour: scripted vs. Machine Learning (ML).

The post Behaviour Comparison appeared first on RIDE.

]]>
Machine Learning

Behaviour Comparison

Overview

This scene demonstrates how one might analyze side-by-side the performance of two approaches for agent cover behaviour: scripted vs. Machine Learning (ML). 

Page Contents

Purpose

This scene demonstrates how one might analyze side-by-side the performance of two approaches for agent cover behaviour: scripted vs. Machine Learning (ML). 

How to Use

The scene consists of two Training Area prefabs divided by agent type:

Agent Type: Scripted

  • Logic: Use cover and enemy positions to determine the best* place to take cover
  • Best = closest point to blue that blocks red’s line of sight

Agent Type: ML – Reinforcement Trained

  • Observations: Player, Cover, and Enemy positions
  • Actions: Move up / down / left / right; Rotate left / right
  • Rewards: Standing in cover
  • Punishments: Small moving penalty

During runtime, each respective agent (blue) immediately proceeds with the goal of reaching the cover point closest to its current position that blocks the enemy’s (red) line-of-sight. The board resets automatically when both agents reach the goal, or when the 15-second time limit expires. 

Click the Reset button at any time to clear the board in the event an agent will clearly not reach the goal in time.

Scene Location & Name

Assets/Ride/Examples/Behaviours/AgentBehaviours/ExampleBehaviourComparison.unity

Setup Requirements 

This specific comparison scene requires the ExampleBehaviourComparison script and two instances of the TrainingArea_TakeCover prefab, which contains the TrainingArea_TakeCover script, floor/walls, cover, goals, Agent (script/ML), Enemy game objects.

ExampleBehaviourComparison Script

First, add this script to an object in your scene to set the Training Area type automatically with the TrainingArea_TakeCover prefabs.

TrainingArea_TakeCover Prefab and TrainingArea_TakeCover Script

Next, create empty game objects named “Scripted” and “ML” and then import the TrainingArea_TakeCover prefab into your scene. Move this prefab under “Scripted” and rename as “ScriptedTrainingArea”. Instance this prefab again, moving under “ML” and renaming as “MLTrainingArea”.

The example scene also includes Canvas and TrainignMenu prefabs to create menu panels and buttons.

Customizing and Extending Scene  

Create a prefab variant of TrainingArea_TakeCover to begin constructing your own training environments. 

Prerequisite, if adding your own mlAgents for a similar comparison structure, these agents must be trained prior. See Training TakeCover ML for more information.

The mlAgent in the example scene is modelling cover behaviour by inference. In this case, the training scene was linked to an external python application over Unity TCP containing user-defined observations created in an academy for which the unit “brains” undergo training.

The Barracuda Neural Net inference library “brain” output file imported into Unity and used for the example scene: Assets/Ride/Examples/Behaviours/MLBehaviours/TakeCover.nn

Unity Machine Learning Agents Information

Unity ML-Agents Tools

Unity ML-Agents How-To

Unity ML-Agents Intro – Blog

The post Behaviour Comparison appeared first on RIDE.

]]>