Recording and Playing Audio in Unity

Just published my Recordable Launchpad app on the iOS App Store and Google Play! Actually, the project was created accidentally when I was testing out Unity In-App Purchase, Google Admob Unity SDK, the “Prepare iOS for Recording” checkbox, and Audio Latency.

Since integrating Unity In-App Purchase and Google Admob are rather straightforward, I would be writing mostly about (the more troublesome parts, which are)

• Serializing and Deserializing AudioClips
• Getting Rid of Audio Latency
• Prepare iOS for Recording and iPhoneSpeakerFix

in this article.

Get Rid of Audio Latency

When playing audio effects in Unity, there are always some 0.1-0.5 seconds of latency, which is extremely annoying. The problem is rather simple to fix, but I actually spent a lot of time searching for the best solution.

One method is to use .wav files instead of .mp3 because there would always be some overheads in decompressing .mp3 files. (However, .wav files are much larger…which is fine for short audio effects though)

The other method is really trivial, but it actually solved the problem for me. Just select Best Latency in Project Settings -> Audio -> DSP Buffer Size, and the job is done!

Nonetheless, some Android devices may still have some noticeable latency due to the variety of audio hardwares on different devices. The only solution to the problem may be writing native Android plugins.

Prepare iOS for Recording and iPhoneSpeakerFix

In iOS, the Microphone.Start() call is always laggy. Fortunately, Unity provides a Prepare iOS for Recording checkbox in Player Settings. Though Prepare iOS for Recording solves the Microphone.Start() latency, it generates another issue: All of the audio would come out from the small earphone on the top of the phone instead of the bottom speakers! This is really irritating since the volume would be extremely low, and the quality of audio would be very bad.

After days of searching, I finally found a nice solution on GitHub: UnitySpeakerFix. This little plugin written by cbaltzer is extremely simple to integrate. Just copy and paste the files and add

1

iPhoneSpeaker.ForceToSpeaker();

after every Microphone.Start() call, and boom! The audio would now come perfectly out of the speakers again!

Serialize AudioClip

In my Recordable Launchpad app, I saved the recorded AudioClips in binary files. Whenever the app is started, the AudioClips are deserialized from the local binary files, and when the app quits, the AudioClips are serialized and saved in the local directory.

The most important points when serializing or deserializing AudioClips are

• Get new AudioClip instances by AudioClip.Create().
• AudioClip in Unity consists of three important fields: frequency, samples, and channels.
• The actual audio samples are stored in a float typed array (float[]) called sample, where the length is channels multiplied by samples.
• Copy audio samples by the AudioClip.GetData() and AudioClip.SetData() functions.

So, firstly, we should create a serializable class to store AudioClip:

1
2
3
4
5
6
7
8

[Serializable]
class AudioClipSample
{
    public int frequency;
    public int samples;
    public int channels;
    public float[] sample;
}

Then, the serialize and deserialize code should be quite simple:

Deserialize AudioClip from local file and set to AudioSource

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

public static void LoadAudioClipFromDisk(AudioSource audioSource, string filename)
{
    if (File.Exists (Application.persistentDataPath + "/"+ filename)) {
        //deserialize local binary file to AudioClipSample
        BinaryFormatter bf = new BinaryFormatter ();
        FileStream file = File.Open (Application.persistentDataPath + "/" + filename, FileMode.Open);
        AudioClipSample clipSample = (AudioClipSample) bf.Deserialize (file);
        file.Close ();

        //create new AudioClip instance, and set the (name, samples, channels, frequency, [stream] play immediately without fully loaded)
        AudioClip newClip = AudioClip.Create(filename, clipSample.samples, clipSample.channels, clipSample.frequency, false);

        //set the actual audio sample to the AudioClip (sample, offset)
        newClip.SetData (clipSample.sample, 0);

        //set to the AudioSource
        audioSource.clip = newClip;
    }
    else
    {
        Debug.Log("File Not Found!");
    }
}

Serialize AudioClip to local file

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

public static void SaveAudioClipToDisk(AudioClip audioClip, string filename)
{
    //create file
    BinaryFormatter bf = new BinaryFormatter ();
    FileStream file = File.Create (Application.persistentDataPath+ "/" + filename);

    //serialize by setting the sample, frequency, samples, and channels to the new AudioClipSample instance
    AudioClipSample newSample = new AudioClipSample();
    newSample.sample = new float[audioClip.samples * audioClip.channels];
    newSample.frequency = audioClip.frequency;
    newSample.samples = audioClip.samples;
    newSample.channels = audioClip.channels;

    //get the actual sample from the AudioClip
    audioClip.GetData (newSample.sample, 0);

    bf.Serialize (file, newSample);
    file.Close ();
}

The full source code can be found on my GitHub