Thursday, February 26, 2015

Video Game Audio - Cool Way to Make a Perfectly Smooth and Seamless Loop


If you work in the gaming industry—whether you're a programmer, a composer, a sound designer, a project director, or anyone else who may deal with game audio—it's almost a guarantee you've come in contact with loops. They drive so many aspects of the gaming experience that even if you aren't the person implementing the loops, chances are that some of the music or sounds you've worked on will be looped at some point in their games.

Because the gaming industry is fueled by highly interactive experiences, your loops most likely will not be handled in a linear format—that is, by copying and pasting or otherwise extending an audio event to match a fixed timeline—as would be the case for loops in film and television. Instead, the game's engine will trigger the looping of various audio files so they can repeat an endless amount of times until something new is triggered to take their place.

For this reason, a smooth transition from the end of a single audio file (or chain of audio files) back to it's beginning is imperative, but creating seamless loops remains one of the trickiest challenges of implementing game audio today. When created, exported, or formatted improperly, an audible pop, a harsh click, a slight gap with no sound, or a lack of sound continuity (such as the reverb tail in a musical loop) will be present.

Current Methods of Making Seamless Loops

While there are a couple of common methods used to attempt to tackle these issues, none are totally perfect or can be used in any situation, and all may require multiple attempts to get an acceptable outcome. I'll review some of these common ways to work with loops and then share a solution for achieving "theoretically perfect" loops that my own experience and roadblocks led me to discover—a method that works extremely well for ambiences and environments and very well for certain ambient music tracks, though with the right amount of work, it could be applied to any loop.

Both sound designers and composers (and all other audio experts) should be familiar with the term "zero crossing." In short, it is the center of a waveform—often represented by a straight line in DAWs. When the wave touches this line on its journey to cross to the opposite side, it is at it's quietest point, or 0db. By finding the point of zero crossing in a waveform and trimming the file to that point, you are able to end a loop with reduced risk for popping or clicking. However, sometimes this means you have to change the length of the file, even if slightly, which can be awkward for rhythmically driven tracks. 

Other times, zero crossing just doesn't line up like you'd hoped and you have to rely on a fade of 5-10 ticks or samples to force out the click that would otherwise be present. Usually, a fade so small is not easily noticeable, but if the track is fairly rambunctious, a similar fade may be required at the beginning of the audio file. These "forced" zero crossings can leave a slight gap in sound, and they of course don't get into the issues of reverb tails, the left over echo-like sounds that simulate a realistic environment. 

If you go into a big hall and shout "echo," you'll hear the reflections—your voice bouncing off of the walls and other surfaces—trail off slowly. If it all stopped the second the you shut your mouth, you would easily notice that something is off. This same unnatural feeling happens when a loop is suddenly cut off at the end in order to keep the timing while transitioning back to the beginning. 

It's not an issue while composing or designing certain sounds because any reverb units you have applied in the project are active and hold out their respective reverb tails when MIDI notes or audio files are played back. So, loops are seamless all the time, every time within the project. However, when you actually write or bounce down all of that information into a single track, the reverb is written along with everything else. There is no active unit gauging when each note is played and waiting to hold them out accordingly. Instead, the reverb of the final moments of the original session carries out beyond the length of the final sound and is written to your new exported file.

Now, in order to maintain that reverb tail's naturalness, you must cut the tail off your exported file and place it on a second track at the beginning of the loop. That poses a few problems in itself, and as renowned game composer Winifred Phillips agrees in her book, The Composer's Guide to Game Music, when done improperly, it can be quite dissonant and displeasing to the listener (p173-174). For this reason, she advises ending and starting loops with the same notes and making sure that reverb times are short enough (at least at the end of the track) to keep multiple notes from overlapping back to the beginning. 

Of course, just as zero crossing fades don't cover reverb issues, reverb edits don't address pops and clicks. Often, the two techniques must be combined to make acceptable loops, and the common result still leaves that tiny, tiny gap in playback. It may not be noticeable in all tracks, but highly reverberated, ambient, or "spacey" loops will tend to stick out like a sore thumb, even for such slight gaps. Luckily for the modern game industry, multiple tracks often play at the same time (ie dialogue, 3 different nature tracks, background voices, clothing and sfx, and music), so a split-second gap in any one loop may be covered by the business of the others. But this isn't always the case. Imagine you are playing a game in which the music is sparse and, because your character is standing still, the only other sound is the light ambience of a cave or abandoned library. Gaps could ruin the experience here.

Another technique that is less preferable but is still present in the game world is to have an ending in the loop, which makes it extremely obvious when the piece starts over due to the intended silence. 

Still another technique is just leaving the file alone and doubling it, either in a DAW or directly in a middleware program like FMOD or WWise. Because middleware is designed for both the composers/sound designers and programmers to be able to share a common template and can integrate sounds into a game's engine, a programmer could then take over at any point. Regardless of who does the work, someone can set a second copy of the file to begin exactly when the first should start over, looping the second file only so that the reverb tail from the first is carried over but not printed on the original track. While that solves the issue of dissonance the first time a loop is played, it still leaves room for pops and presents a bigger issue of files potentially being twice the size. That can really add up considering game engines can only hold so much at once.

A Newer Method for Making Seamless Loops

Earlier, I mentioned that my method is "theoretically perfect," but because of reverb tails and certain melodic writing styles, it is not yet perfected for music…and since reverb tails are so important in original musical sessions, it may never be perfected for music. However, it does the three other things that may be hard to come by in any other method, still making it a gem and viable option for many musical tracks. 

So, here is what I discovered along with its pros and musical cons:

When you are doing the regular editing of audio files in your DAW of choice, you sometimes need to cut or separate them into two or more pieces. However, if you don't move those pieces after they're split and initiate playback, you'll find that they flow together seamlessly, as if they were still a single file. I noticed this in the first song I ever edited, but it wasn't until the spring of 2014, when I was tasked with transforming some existing environment tracks for a game into loops, that I began to think bigger. 

These pesky tracks had all sorts of weird high points and low points, and nothing was ever consistent enough to just have them start and finish. The audio director knew this and said we were to follow the common method used by his team of layering them all with different starting points and ending points so that any time one faded out, the others would still be going. That way, no one would ever know that several individual tracks were constantly fading in and out. I still presented a version this way, but I remembered how important file size and the amount of files per project were to the development process. 

That's when the idea hit me! If regular audio tracks could play seamlessly even when cut in half, why couldn't loops work in the same way? The logic behind the thought said, almost counterintuitively, that rather than searching for near-zero crossing points that are closely lined up to loop with, why not loop anywhere by reversing the order of two split regions, regardless of wave positioning?

So, on my own time, I pulled up one of the files on my computer. I created a new project in my DAW, loaded it, then looped it. The change from end to beginning was abrupt. Then, I cut it in half. It still sounded like one single track, as I was expecting, with the exception of the abruptness of the loop point. I excitedly switched the order of the two halves, so that the second half played first and the first half immediately followed.

The result was the same! My file started in the middle of the environmental atmosphere, abruptly changed halfway through, and then looped back through the beginning as if it was a single file. That's all I needed to know for my experiments to begin.

I quickly undid everything and shortened the file on both ends so I could have some leftover room for crossfading. Then, I cut it more carefully, looking for a point closer to zero crossing and perhaps more importantly, listening for a low point in environmental activity (because unique sounds will cause clicks when interrupted, even if there is no huge spike in the waveform).

When I flipped the order of the two files this time, the change near the middle was still noticeable, but much less abrupt. At that point, I added a small crossfade. When an ambience is very consistent throughout, that's all that is required, and it worked on some of the tracks I played with later. But not here. The buzzing still felt different. This was a wild track with monkeys, birds, bugs, tree branches, and all sorts of things happening as it progressed. So, I made a large crossfade, which slowly and unnoticeably transformed the various buzzes, chirps, and calls into different buzzes, chirps, and calls. In fact, anything like chirps and calls are never too big of a deal because the fades make them just seem quieter. That may even be desirable to further differentiate them from similar sounds.

The key here was getting the subtle changing of the buzzing frequency to feel like it wasn't suddenly changing. After all, it had gradually risen in pitch throughout the original file, so my goal was to slowly lower it back to the first pitch I heard. A short fade couldn't do that, but a long one did the trick! By extending the length of the fade, the change in buzz pitch became much more natural, and the middle of my file was now seamless. So, when I played the track back a final time, the result was an unending loop that never popped or clicked, never had moment of silence, and used crossfades in the center of the file as opposed to fadeins and fadeouts on the ends. 

The director was of course pleased with my innovation and allowed me to implement the techniques into the project. The result allowed us more consistent environments as well as the ability to reduce and condense files further when required.

That's why I believe this is sheer gold to game sound designers. Not only can you condense tracks that don't need to act independently into a single file, you don't necessarily have to worry about reverb tails either. Yes you could add your own reverbs to looped ambiences and atmospheres in your DAW, but then they are written forever. Developers have much greater control over those loops if you create them dry and allow the engine to trigger the appropriate reverbs through middleware. Then, if a swirling orb follows you out of a cave into a forest and finally through the front door of a small cottage, the reverb can automatically change to reflect those environments, and the same reverbs can be applied to all sfx for uniformity.

Either way, reverb tails are much less off-putting at the beginning of an ambience should you need to write them to the track. This differs from music because different instruments or instrument groups may all have different types of reverb and different amounts applied in order to best suit the composition. Often, ambient tracks only require a single reverb at a time to reflect the space in which they are sounding, as was outlined in the orb example above.

How this Method Relates to Music

If you are primarily a composer, perhaps you can see some issues that may arise. A lot of them pertain to looping in general, but let's address them all anyway and then explore some solutions.

The first and biggest concern: why would you cut your music in half and start in the middle? Great point. Don't. I wouldn't either. There are at least two ways around this, though one seems way too difficult and is untested. The other is to change the order of your composition before you export it. Pick a good spot to have as your "alternate" beginning, and then move everything that is before it so that it starts after the last bar of your intended ending. Bounce the file. Keep in mind that, just like all of your other methods, this will never work if you are trying to loop a composition that doesn't start and end the in a similar way or if you are trying to make a shortened loop out of a longer piece. A loop has to be prepared to flow back into its beginning from the end, so two wildly different sections will end up just like my bug problem explained above.

By moving your track's beginning to the end before bouncing, you essentially get two reverb tails—the one you wanted at the beginning of your track (if you have been working with one of the common methods mentioned above) and the one that you still need for the middle of your piece (which is now temporarily at the end). Because you are working with loops, your DAW should have fixed tempo rates and a grid that perfectly aligns the sections to the tempo. It's cutting time! Cut where your piece should start and move the two sections of your newly bounced file back into the correct places. Now, you'll notice that the tail from the first half overlaps the second half, so you can move one of the two pieces to a second track and fade away the any pops in order to blend them perfectly.

That brings us to our a couple of myth-related issues: what if the track is highly rhythmic and ends with a downbeat on the one? Well, unfortunately my friend, you will have two hits on the one count overlapping each other. This shouldn't be an issue though and here's why. If it's looping, you shouldn't really be bouncing an ending downbeat to begin with. It's fine for the soundtrack so that fans know the song is over, but it would get cut from a loop. Remember how I said don't make a shortened loop out of a longer piece. That's the other reason why. Then you will have extra information you can't get rid of, making crossfades useless. The only solutions, if you have a downbeat ending anyway, are to cut (or more accurately trim and fade juuuust right) the one without the reverb tail, though I suppose you could get creative by cutting the other hit and leaving the tail or automating both to lower volumes if the hits aren't identical (phasing issues would otherwise occur), or you can find a new way to end the piece.

The other rhythm and percussion-related issue has to do with embellishments such as cymbal swells, chimes, bells, etc. that make this method and loops in general harder or impossible to work with. These ring out themselves in addition to whatever reverb tails are produced and therefore must be able to sound at the beginning of the track. Again, if you loop playback while composing, you won't notice that your final cymbal swell is off because the reverb in the DAW will actively carry the tail over to the beginning of repeated loops and it won't play it the first time. As soon as you bounce it though, you'll have a reverb tail that also includes the pinnacle of a cymbal swell. Moving that tail to the beginning of your piece following any method will be awkward since you don't want half a cymbal swell the first time the track plays. If you really need such embellishments at the end of a piece, you may consider seeing if your development team can trigger that sound separately within the engine, though rare is the occurrence that this will be agreed to. On a side note, this is also why loops don't usually feature upbeats, especially melodic upbeats. They have to be structured so differently, and melodic, upbeat reverb is a nightmare to deal with outside of clever middleware usage.

Now that we are back around to reverb tails, let's discuss the other and only important issue they may have: clicks or pops the first time only. In other editing methods, the first loop is great but the subsequent loops are subjected either to no tail, brief silence, or pops and clicks. In this method, those subsequent times are great, but the triggering of the initial music file may be the only point that causes a pop or click because of the written reverb tail. This issue would still be present in other methods, but you have the ability to add a fade just to the reverb tail because it is on a separate track. You may or may not want to keep that tail separate from the technically seamless part of the loop for this reason (though you are then combining methods and giving yourself a lot more work since that probably involves bouncing wet and dry versions of everything).

If you are able to just draw out the reverb tail's initial click with an advanced DAW pencil tool, great. Or, since music, especially when played live, has all sorts of subtle noises in it, you may be able to leave a quick, quiet click as the beginning of a track is the only place that listeners usually are unprepared to pay attention to the details. I've certainly heard them before…only subsequent loops still had the problem. 

Don't fear though; there are two other places for hope. A ridiculously small fade on the front end only may be just enough to keep the file from sounding like there is a slight pause. However, I'm often not a fan of this method because it doesn't work for all of the tracks out there. 

The other method you have less control over, but it is the most ideal situation. Often, game engines will fade in tracks in order to better segue from various locations or gameplay ideas, such as combat and free exploration. If that's the case, the initial pop will be eliminated because the track will start at 0db or will be crossfaded to "resume" from a later point in the loop thanks to the game engine. The really nice thing about this is that even dissonant reverbs can be more acceptable at the beginning of the piece because they will be gone before the full volume of the track is in place. Don't rely on engine fades to save a poorly executed piece though. Only use them when it works with the vision of the game.

So, what about ambient music tracks? 

Those issues listed above you may see with tracks that are highly structured, rhythmic, and melodic or that have huge amounts of reverb copied over to the beginning. Luckily, structured, rhythmic tracks tend to be the easiest to loop with common methods. However, ambient tracks can be trickier when trying to loop them under these methods. But, as they can be created closer to sound design elements and inhabit the "background" by nature, there is great potential for them to be arranged according to this new method with little difficulty or effect on the track. In addition, their ebbs, flows, swells, and quiet moments make them able to potentially start anywhere, and some even offer space for reverb to decay on its own significantly before the next movement starts.

I was recently messing around with an ambient track consisting of various string instrument chords to help further illustrate this point. I opened a new session at the correct tempo and imported the piece. I was then able to move chord progressions around and fade them into each other without any pop or click at the beginning of even the first playback thanks to strings having a slightly delayed attack (the amount of time it takes for a sound to amp up to its desired volume after the note is initiated). 

Because "ambient" can be a vague stylistic term by nature, your ambient track may feature creepy sounds or spacey synths or woodwinds and horns or even warped percussion, so you may have to experiment on your own a bit to find the right spot to split the file. However, you may also find that there is no need for the piece to start from the originally intended beginning, thus saving you the task of having to bounce and flip the tracks again.

Some Final Notes
  • Users of Logic Pro X should be aware that it's loop feature by default adds a few milliseconds to playback. I haven't changed mine, even though I primarily use that DAW for MIDI and scoring, but I'm pretty sure I recall seeing the option to turn this off in the preferences. Pro Tools and most other DAWs shouldn't encounter this issue. Part of the reason this option comes with the newest Logic Pro is because people often choose to humanize MIDI tracks, and some of the quantization options causes notes to sound slightly earlier or later than exactly on tick 1 of beat 1 of measure 1. With the extra few milliseconds, playback can anticipate those notes that are just outside of the actual looped region and still trigger them so you can hear your loops as you're supposed to while composing.
  • This also means that you must watch out for humanized notes. The beginning of a track cannot have notes that start before the first measure or they won't be included in a bounce, notes shouldn't humanized to a position outside of the beginning or end of their regions or they can be cut off, and wherever you intend to make your cut in the bounced file should keep both of these in mind as if that really is the beginning of your track. In fact, it is good to keep notes at the beginning of a Logic score or region at least a few ticks after the first beat.
  • Mp3s add time to the end of loops that some game engines can't account for. This will cause a slight pause in even the perfect loop. Unless your track is a one-shot, it's usually better to convert your wave and aiff files to ogg vorbis unless the developers ask otherwise, will do the converting themselves, or plan to implement full-sized uncompressed files.
  • Always work with a grid and tempo information, and deal with bounced files in a separate project. If the final files aren't perfectly aligned to a grid, loops will become staggered.
Thanks for reading! If you'd like to hire Nate Combs to work on your game project, visit his website at www.natecombsmedia.com or contact him directly at natecombsmedia@gmail.com.

No comments:

Post a Comment