VHToolkit 2 Preview > Download and Set Up Source Code

Download and Set Up Source Code

Disclaimer: Preview version; content and features subject to change. 

Overview

This tutorial is geared towards any researcher and developer who wants to customize and expand upon the VHToolkit 2.0 project.

Page Contents

RideUnity_LevelSelect

Overview

The VHToolkit 2.0 example project source code showcases the architecture and key capabilities of the project. 

Source code is available for the following platforms:

  • Win64
  • macOS
All use is governed by the License Agreement. 

Requirements

Hardware

Refer to Unity documentation for Editor supported systems.

Obtaining the Source Code

The process is to acquire the VHToolkit 2.0 source code is as follows: 

  1. Submit a VHToolkit 2.0 source code request with desired platform to vhtoolkit@ict.usc.edu.
  2. Download links:
  3. Use the provided password.

Installing the Source Code 

Instructions for download and setup of VHToolkit 2.0 source code.

Windows

  1. Download the package to your local PC (20 GB or greater of free space recommended) 
  2. Run the installer
  3. Review the license and accept terms
  4. Install the files to your preferred location

Note, no shortcuts or uninstaller included; users must manually delete the VHToolkit2 source code directory. 

Mac

  1. Download the package to your local PC (20 GB or greater of free space recommended) 
  2. Extract the compressed folder contents at your preferred location
  3. IMPORTANT: Due to macOS security features, the OS may block certain dependencies at run-time; open a terminal window at the IVA project folder
  4. Input the following to edit the file attributes: xattr -cr “Assets/Ride_Dependencies (local)/SpeechSDK/Plugins/MacOS/libMicrosoft.CognitiveServices.Speech.core.dylib”
  5. Input the following to edit the file attributes: xattr -cr “Assets/Ride_Dependencies (local)/Oculus/Oculus/LipSync/Plugins/MacOSX/OVRLipSync.bundle”

Prerequisites

  • Unity Hub
  • Unity version 2022.1.12f1
  • VHToolkit 2.0 project source (requires minimum 20 GB of free disk space)

IMPORTANT

Note, under 2020.3.x, Microsoft.Extensions.Logging.Abstractions.dll and NonverbalBehaviorGenerator.dll need to be manually excluded from being compiled by unchecking their respective Import Settings inside the Inspector pane.

Recommendations

  • Version control software (Subversion client TortiseSVN or similar)
  • Text or source code editor (Notepad++ or similar)

VCS Subversion Client – TortiseSVN

Version control software (VCS) allows the tracking of individual file changes inside of a code base. In the event a “breaking” code change is committed, these changes can be easily compared against prior versions of the file(s) using the VCS client, and the file(s) reverted to another “known good” version.

  1. Navigate to https://tortoisesvn.net/
  2. Download, install and configure latest TortiseSVN Subversion client
  3. Note, TortiseSVN is Windows only; alternative Subversion clients exist for macOS and Linux 

Text or Source Code Editor – Notepad++

Advanced text editors tailored to script and source code will help avoid mistakes that can occur when editing these types of files in a standard text editor or word processor.

  1. Navigate to https://notepad-plus-plus.org/
  2. Download and install latest Notepad++
  3. Note, Notepad++ is Windows only; suitable alternatives exist for macOS and Linux

Unity Hub Installation

The Unity Hub utility controls which versions of the platform are installed on the local machine.

  1. Navigate to https://unity3d.com/get-unity/download
  2. Click “Download Unity Hub” button.
  3. Download and install Unity Hub.
  4. Run the application, and proceed to create an account or login.
  5. Inside Preferences, License Management, choose Activate New License option.
  6. Complete License Agreement.   

Refrain from acquiring any version of Unity from the Hub for the moment. See next section for Unity installation steps.

Windows – Unity Editor Installation Using runUnity.bat 

With Unity Hub now installed, locate the runUnity.bat file at the top level of the IVA project. When first initialized, this process calls Unity Hub for a “headless” (background) automatic installation of the proper version of the Unity editor and modules for the project. (Remain connected to the Internet during the next set of steps.) 

  1. Recommend for first-time launch with runUnity.bat that it’s initialized through the command line; using this method displays the status of the process and any errors.
  2. On Windows, open File Explorer to the directory with runUnity.bat.
  3. Click in the address bar and clear the path text.
  4. Type “cmd” in the field and press Enter; a command prompt will appear with the current directory path.
  5. Type “runUnity” and press Enter.
  6. Wait for the “headless” process to download and install the Unity target version via Unity Hub; this can take a while.
  7. When complete, Unity will initialize and begin to ingest the project files and build a local library; this too can take a while.
  8. After the local library creation finishes, the Unity application will open to the IVA project.
  9. If successful, subsequent launches of the project can be done with runUnity.bat directly, no need to use the command line.

Important

Always use runUnity.bat to open the RIDE Unity development environment. This will ensure you are using the correct version of Unity every time.
 

Tip

If the “headless” installation process fails, the desired version can be acquired through the Unity Archive page, https://unity3d.com/get-unity/download/archive

  1. Select the option from the page to download and install the desired version through Unity Hub; installation can take a while.
  2. Note, Unity Hub will prompt for a license if not already present; create an account or login, and then complete the License Agreement process.
  3. After installation completes, relaunch runUnity.bat in order to initialize Unity and create the local project library; this too can take a while.
  4. Once the local library creation finishes, the Unity application will open to the IVA project.
 

MacOS – Unity Editor Installation via Unity Hub 

With Unity Hub now installed, the desired version can be acquired through the Unity Archive page, https://unity3d.com/get-unity/download/archive

  1. Select the option from the page to download and install the desired version through Unity Hub; installation can take a while.
  2. Note, Unity Hub will prompt for a license if not already present; create an account or login, and then complete the License Agreement process.
  3. After installation completes, from the Unity Hub, choose Open, Add project form disk.
  4. Select the local IVA project directory.
  5. Click its entry to launch the project.
  6. Unity will create a local project library with first launch, which can take a while.
  7. Once the local library creation finishes, the Unity application will open to the IVA project.

Using the Source Code

LevelSelect

A simple LevelSelect scene serves as a hub to launch the example scenes for the VHToolkit 2.0 project.

Location: /Assets/Scenes/LevelSelect

  1. The main menu presents the following choices:
    • VH Sandbox: Main example scene of the VHToolkit 2.0, showcasing how characters can be configured, instantiated, and edited
    • VH Sandbox – Minimal: Reduced set of services to that of only non-Windows based cloud services
    • ExampleNLP: Compare NLP services
    • ExampleLLM:  Compare LLM models
  2. To exit a scene, press the Escape key to return to the main menu
  3. Close the LevelSelect by pressing Escape key

Scene Information and Instructions

Visit the other tutorial pages for specific scene details and instruction.

General Controls

When in a particular scene, the following controls are commonly available: 

  • J key toggles mouse look and WASD “flying” camera
  • ~ key toggles the console tool

GUI Debug Menu Navigation

Certain scenes utilize a debug menu that appears in the upper-left corner by default. 

  • If enabled, C key or F11 toggles off/on this menu
  • Specific scene functions always appear in the first menu
  • Step to the next menu be either clicking the middle title bar or arrow (>) at the top; click the double-arrow (>>) to expand the console
  • System statistics appear in subsequent menus

Known Issues and Troubleshooting

Windows

Uninstaller
  1. Uninstaller fails to delete Start menu and desktop shortcuts; please remove manually with uninstall of VHToolkit 2.0
Windows speech recognizer unavailable
  1. Ensure under Settings > Speech,  Online speech recognition option is switched to On

Mac

App or dependencies blocked from running by versions of macOS

Due to macOS security features, the OS may block certain dependencies at run-time. Use the following workaround for any prompts received:

  1. Open a terminal window at the IVA project folder
  2. Input the following to edit the file attributes: xattr -cr “Assets/Ride_Dependencies (local)/SpeechSDK/Plugins/MacOS/libMicrosoft.CognitiveServices.Speech.core.dylib”
  3. Input the following to edit the file attributes: xattr -cr “Assets/Ride_Dependencies (local)/Oculus/Oculus/LipSync/Plugins/MacOSX/OVRLipSync.bundle”
  4. Play the VHSandbox – Minimal scene again
Speech input and camera feed not available
  1. If prompted by the OS, enable access to your microphone and webcam

General

Config file out of date and errors when running any example scene 
  1. Update the config file by launching LevelSelect > ExampleLLM scene
  2. Open the debug menu:
    • Windows, press F11 key
    • Mac, press Cmd+F11 key, or option + F11; note, may need to enable the option, System Preferences > Keyboard > “Use F1… as standard function keys”
      1. MBP touch bar reference: Apple Support
  3. Config menu appears; if not, click header or arrow of debug menu
  4. Click Reset to Defaults button
  5. Return to LevelSelect and choose desired scene again
Services unavailable and various errors when running any example scene 
  1. Ensure PC has an active Internet connection; majority of all services require Internet except for “local” or “external” Microsoft/Windows services

Example Scenes

VHSandbox
  • AWS Lex v1 selected as both NLP Main and NLP Sentiment will fail and soft-lock input section of UI; as workaround, create new VH, then delete the problematic VH
    • Also occurs with NLP Main as AWS Lex v1 and NLP Sentiment as AWS Lex v2; as workaround, do not use AWS Lex as options in both fields
      • Best practice if using AWS Lex, populate both fields with AWS Lex v2
  • Character utterance audio may repeat and not match text for ElevenLabs voices
  • Delay from click & hold until mic input status change to “listening”
  • Mic input may fail to function with initial click & hold
  • SeaView environment locations run very slow on lower-end hardware
  • Rocketbox characters do not display face mirroring
  • Mobile Speech Recognizer does not function on desktop systems
  • Webcam view may fail to update with creation and/or deletion of multiple characters
  • AWS Polly TTS, corresponding TTS Voice options may fail to populate
  • Windows TTS (External Windows Process) may cause strong head nod at start of utterance for characters
  • NLP Sentiment as AWS Lex v1 or v2 and TTS as ElevenLabs TTS causes VH unresponsiveness and soft-lock input section of UI
  • Rocketbox characters with NLP Main set as AWS Lex v1 or v2 and TTS  as ElevenLabs TTS may cause first utterance to repeat twice
  • Binary only: NVBG System (Local Integrated Class Library, Windows Only) causes VH unresponsiveness and soft-lock input section of UI
  • Renderpeople Test character does not lipsync
ExampleNLP
  • Google DialogFlow may not function (errors present) on certain versions of macOS
  • Microsoft Azure Virtual Agents (PVA) service disabled
  • “Entities” may not populate depending on service selected

New Scene Creation for LevelSelect Scene Listing 

The VHToolkit 2.0 IVA project is built upon the modular architecture of the Rapid Integration and Development Environment (RIDE) platform.

Users will want to ensure their development environment is completely setup before continuing further.

How to create a basic test scene compatible with LevelSelect scene listing

If new to Unity and RIDE toolset, recommend first creating the most basic test scene that can launch from, and return to, the LevelSelect scene listing. 

  1. Unity editor, File > New Scene
  2. File > Save As…
  3. Within the project directory, navigate to /Assets/Scenes
  4. Create a new sub-folder, giving it the same name as the new scene that will be saved. (Ex. “ExampleTest”)
  5. Open this new sub-folder.
  6. Name the scene the same as the sub-folder and click Save.
  7. In Project panel, search field, type “ridebaseminimal”
  8. Double-click this file (RideBaseMinimal.cs) to open it in the system default text editor. Recommend Notepad++ or other source code editor.
  9. Copy the first 31 lines of file text.
  10. Create new file in text editor of choice.
  11. Paste 31 lines into new file.

  12. Modify the script to appear exactly as the provided code block example; however, replace the “public class” value of “ExampleTest” with the name of your newly saved scene. This new class is derived from “RideBaseMinimal” which enables the bare minimum of  RIDE systems, including load/exit of the individual scene within the LevelSelect scene.  

    ExampleTest.cs
				
					
using System;
using UnityEngine;
using Ride;
using Ride.IO;
using Ride.UI;

namespace Ride.Examples
{
    public class ExampleTest : RideBaseMinimal
    {
	}
}
 
  1. Save this file to the same sub-folder inside of /Assets/Scenes named after the new scene, which also contains the scene file itself.
  2. Unity editor, in Hierarchy panel, right-click and choose Create Empty from the context menu.
  3. Rename object to match the scene name.
  4. Ensure object selected.
  5. Inspector panel, click Add Component button.
  6. In the search field, type the name of newly saved script, which matches the object/scene name.
  7. Click the script result to add it to the object as a component.
  8. In Project panel, search field, type “ridesystems”
  9. Drag the corresponding prefab into the scene Hierarchy. This prefab initializes all the systems available in RIDE. Expand the prefab in the Hierarchy view to disable systems as desired.  
  10. Save scene.
  11. Verify the scene works by clicking the Play button in the top-center of the Unity toolbar to compile and launch the scene.
  12. Open the Console panel to watch for any errors/warnings. If an error appears, double-check the content of the newly saved script file.
  13. If the scene runs successfully, stop the editor run-time mode by clicking the Play button again.

Building and Running a Local Standalone Application

Configuring LevelSelect scene in Build Settings

Unity provides various options to configure a standalone application prior to building locally.

  1. Unity editor, choose File > Build Settings…
  2. Under “Scenes in Build” ensure all scenes desired for the standalone application are checked.
  3. Platform options, use default: PC, Mac & Linux Standalone.
  4. Ensure Architecture property set to x86_64.
  5. Note, additional properties are linked from the “Player Settings…” button.

Building and running local LevelSelect standalone application

  1. When ready to build the local standalone application, return to File > Build Settings…
  2. Click Build button.
  3. In the build location window, navigate to one level outside of the current project folder. Depending on local file structure, this could be outside of /IVA.
  4. Create a new folder, and name it “bin”.
  5. Open “bin” and create a sub-folder with a unique name.
  6. Ensure this folder is selected, and then click Select Folder.
  7. Unity build process will initialize, and display a “Building Player” dialog.
  8. Once finished, the /bin unique sub-folder will open in a new window.
  9. Run the “VHToolkit-IVA2023” executable.
  10. Verify the LevelSelect menu appears, and any selected scene from listing loads as expected.

Unity Reference and Training

Resources to build familiarity with the Unity editor and application development.

Unity Manual, Version 2022.1

Unity Scripting API, Version 2022.1

Unity Learn Premium, Getting Started With Unity

Unity Learn Premium, Tutorials

The project/effort/work depicted here was or is sponsored by the U.S. Army Research Laboratory (ARL) under contract number W911NF-14-D-0005. Statements and opinions expressed and content included do not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.