Game Engine Anatomy 101 May 17, 2002 Game Engine Anatomy 101, Part VI By?Jake Simpson Discuss this now (26 posts) Sound and music in games are becoming increasingly important in recent years due to both advances in the game genres people are playing (where sound is an actual game play feature, such as the aural cues in Thief and other games of that ilk), and in technology. Four speaker surround systems are now both affordable and commonplace in the gamer's arsenal. Given spatialization of sound, obstruction and occlusion of noise, and the dynamic music many games employ these days to heighten the emotional responses of the player, it's no wonder that more care is being given to this area. Right now in the PC arena, there is really only one card of choice for gamers – the Sound Blaster Live!. From old time PC sound card manufacturer Creative Labs. Creative has provided their EAX sound extensions for DirectX for a number of years now, and they are a founder of the new OpenAL (Open Audio Library) initiative. OpenAL, as it sounds, is an API for sound systems in the same way that OpenGL is a graphical API. OpenAL is designed to support a number of features that are mostly common across sound cards, and provide a software alternative if a specific hardware feature is not available. For a better definition of OpenAL, I asked Garin Hiebert of Creative Labs for a definition: "Borrowing from our "OpenAL Specification and Reference" here's a definition: OpenAL is a software interface to audio hardware, providing a programmer with the ability to produce high-quality multi-channel output. OpenAL is foremost a means to generate audio in a simulated three-dimensional environment. It is intended to be cross-platform and easy to use, resembling the OpenGL API in style and conventions. Any programmer who is already familiar with OpenGL will find OpenAL to be very familiar. The OpenAL API can easily be extended to accommodate add-on technologies. Creative Labs has already added EAX support to the API, which programmers can use to add sophisticated reverberation, occlusion, and obstruction effects to their audio environment. " Soldier of Fortune II features this new system, as does Jedi Knight II: Outcast, along with the Eagle world/sound properties editor. What is Eagle? Before we get into that, let's discuss a couple of other systems, and define some sound terms. click on image for full view Miles Sound System Another system out there is the Miles Sound System. Miles is a company that produces a plug-in to your own code that will handle all the necessary talking to specific sound cards (like the Sound Blaster Live! series, or an older A3D card for example) while getting the most of each card. It's very much like an API front end, with extra features bundled in. Miles gives you access to things like MP3 decompression amongst other things. It's a nice one-stop solution, but like everything, it costs money as well as being one extra layer between your code and the hardware. For fast sound system production though, it's very useful, and they've been around for a while, so they do know their stuff. Sound Terms Let's start with obstruction and occlusion. They sound the same, but are not. Occlusion means basically that the listener has some encompassing obstacle between them and a sound being played. click on image for full view NOLF 2 Let's say you hear bad guys inside the house in this screenshot from NOLF 2. You can hear them, but their sounds are pretty muffled and muted. Obstruction is similar, but the obstacle between you and the sound is not encompassing. A good example of this is having a pillar between you and the sound source. You are still hearing the sound due to echoes in the room, but it isn't the same as the sound coming directly to your ears. Of course this does rely on knowing what is in a direct line between your ears and the sound source. And the required processing can get pretty time consuming depending on the size of the room, the distance of the sound source from you, and so on. We will talk about traces later-- suffice to say it can often be the reason for slower frame rates. The A3D code inside of Quake III does this stuff, and frame rates can often be improved by turning these options off. Tribes 2 is another sufferer from this malady. Turn off the 3D Sound options and your frame rates instantly get better, which makes sense when you consider how big the Tribes worlds are, and how far you can see. Next are sound material properties. Most sound cards give you the capability to modify sounds being played by using definable filters to act on that sound. For instance, there is a big difference between hearing a sound underwater, or in a cloth-covered room, or in a long corridor, or an opera house. It's pretty cool to be able to change the way you hear a sound depending on the environment you are in. click on image for full view Eagle Back to Eagle… This is an editor that allows most first-person shooter map designers to import their maps into the tool, and then construct simplified geometry that creates a sound map for EAX code in the actual game engine. The idea is you don't need the complex geometry of a real graphical map to simulate sound environments. You can also assign sound materials to the resulting simplified map so sound environments can change dynamically. I was witness to a demonstration of this on Soldier of Fortune and Unreal Tournament and it really is quite striking. Hearing all the sounds change when you plunge into water is a very immersive experience. OK, let's move on. For the consoles, you are more limited in your possibilities because of static hardware – although both on the PlayStation 2 and the Xbox this hardware is pretty damn cool. When I say limited, I only mean in expansion, not in what it's capable of doing. I wouldn't be the least bit surprised to see games with Dolby Digital 5.1 output very shortly from these consoles. Xbox, with its MCP audio processor can encode any game's audio into 5.1, and the game doesn't need to be specially coded to take advantage of this feature. Dolby has brought ProLogic II to the PS2, and has teamed up with Factor 5 to enable ProLogic II for GameCube games. On Xbox, game titles Halo, Madden 2002 and Project Gotham Racing all have 5.1 Dolby Digital audio content. DTS also recently unveiled an SDK for PS2 game developers to bring a reduced bit-rate version of DTS audio to games on that platform. Positional Sound--It's a Complex World Now there are some issues with sound spatialization that not many have dealt with. I speak of putting sound in a real 3D world. Having four speakers around you is a great start, but it is still only in two dimensions. Without speakers above you and below you, you really aren't getting 3D sound. There are some sound modulation filters that attempt to address this, but really there is no substitute for the real deal. Of course most games are really only played in two dimensions for the most part anyway, so it's not such a big deal. Yet. One of the most important features of any sound system is actually the mixing of the sounds together. Once you've decided what sounds you can actually hear based on where you are located, where the sounds are in space, and what the volume is for each sound, then you have to mix the sounds. Usually the sound card itself handles this, which is the primary reason for the existence of the card in the first place. However, some engines out there decide to do a 'premix' in software first. That doesn't make much sense really, until you look at a bit of history. When sound cards first came out there where many different approaches to mixing. Some cards could mix 8 sounds together, some 16, some 32, and so on. If you always want to hear 16 possible sounds, but you don't know if the card can handle it, then you fall back to the tried and tested route-- that of doing the mixing yourself in software. This is actually how the Quake III sound system works, but begs the question: "Quake III was released into a world of A3D and Sound Blaster Live! Cards, which are more standardized than ever, so why do this?" That's a good question. The sound system for Quake III is actually almost line-for-line the same sound system that was in Quake II. And Quake I, and even Doom. When you think about it, up until A3D cards and the SB Live! Card, sound systems hadn't really changed in requirements for years. Two speakers, two dimensions, and a simple volume drop off for distance. From Doom through Quake III not much has changed. And in the games industry, if it ain't broke, don't fix it. click on image for full view Miles Sound System Usually you would just use DirectSound to do your mixing for you, since it would detect the sound hardware available, or fall back to software, much as DirectX does for 3D cards. In 90% of sound cases, a software fall back doesn't really make that much difference to your frame rate. But the Doom engine was created when Direct Sound wasn't even a glimmer in some mad coder's eye. It never got updated because it never really needed to. Of course, you could use some of the clever features of the SoundBlaster Live! Card, such as a room's echoic characteristics: a rock room, or an auditorium, a cavern, a football stadium, etc. And you really should use the mixer provided by the hardware, after all, that's what it is there for. One drawback of this approach is that often the result of the mix isn't available to the program itself, as it's being done inside the card rather than in main memory. If you need to look at the resulting volumes for any reason, you are out of luck.

We haven't talked much about music generation in games. Traditionally there are two approaches, one being a straight music .wav file (or equivalent). It's pre-made, ready to run, and a minimum of fuss. However, these are expensive in terms of memory and playback time. The second approach is to code up a MIDI track using preset samples. This is often cheaper in memory, but has the drawback of having to mix several sounds together at once, thereby using up sound channels.

Dynamic music is the ability to change your music depending on the action being witnessed in the game, such as slow music for exploration, and fast for combat. One thing that's hard to do with pre-made music is to beat-match it so you can fade from one piece of music to another, which is easy with a MIDI track. Often though, if you do the fade fast enough, or fade one down before bringing up the other, you can get away with it.

click on image for full view

Cakewalk's Sonar XL Composers use tools like Cakewalk's Sonar XL to create game music soundtracks.

By the way, before we leave this subject it's worth mentioning that companies exist that specialize in creating purpose-composed music for your games. The FatMan (www.fatman.com) is one such company. It's probably easier to contract out music than anything else, which is how they exist.

Lastly of course, the in-thing for games right now is the MP3 format, which allows massive 11-to-1 compression of sound samples, yet only takes a fraction of the CPU time to decompress them before being thrown at a sound card. When I was at Raven Software, with Star Trek Voyager: Elite Force we managed to get three complete languages on one CD by use of MP3, and still have space left over for more graphics. Mainly, we MP3 only for the non-player characters (NPC) voices, since streaming an MP3 and decompressing it on the fly for all of the game's audio effects was more than the hardware could handle, although it's a definite possibility in the future. Newer formats like AAC from Dolby and WMA from Microsoft offer equal or superior audio quality at nearly twice the compression rate (well, actually half the bit rate) of MP3, and may find their way into future game titles.

That's it for this segment. Next up will be developing for networked and online gaming environments.

參考譯文：

????????????????????????????????????????????????????????????第 6 部分 聲音系統，音頻APIs
聲音系統
由于人們玩的游戲在種類和技術上的進步，聲音和音樂近幾年來在游戲中正逐漸變得重要起來（聲音是一個實際游戲的可玩特點，比如在Thief和其它同類游戲中的聽覺提示）。現在四聲道環繞系統在游戲玩家的寶庫中是負擔得起的和平常的事。給定空間的聲音，噪音的障礙和閉塞，和動態的音樂，如今許多游戲使用這些提高玩家情緒上的反應，更多的關注投入到這個領域就不足為奇了。



現在在PC競技場中，游戲玩家實際上只有一種聲音卡可以選擇 -- PC聲卡制造商創新公司（Creative Labs）的Sound Blaster Live！ 從舊的時間個人計算機聲音卡片制造業者有創造力的中心. 多年來創新公司已經為DirectX提供了他們的EAX聲音擴展，并且他們是發起新的OpenAL（開放音頻庫Open Audio Library）的創立者。就如同OpenGL是一個圖形API一樣，OpenAL，像它起來聽一樣，是一個聲音系統的API。OpenAL 被設計為支持大多數通常聲卡的許多特征，而且在一個特定的硬件特征不可得時提供一個軟件替代。



為了更好的定義 OpenAL，我向創新公司的Garin Hiebert詢問了其定義:

"這里借用我們的 " OpenAL 規格和叁考" 的一個定義:

OpenAL 是對音頻硬件的一個軟件接口,給程序員提供一個產生高質量多通道輸出的能力。OpenAL 是在模擬的三維環境里產生聲音的一種重要方法。它想要跨平臺并容易使用，在風格和規范上與OpenGL相似。任何已經熟悉OpenGL的程序員將發現OpenAL非常熟悉。

OpenAL API能容易地被擴展適應插件技術.創新公司已經把EAX支持加入到這套API了，程序員可以用來給他們的聲音環境增加復雜的反響，比賽和障礙效果。

如同Jedi Knight II: Outcast 一樣，連同Eagle 世界/聲音特征編輯器,Soldier of Fortune II 以這個新系統為特征。什么是Eagle？ 在介紹這個以前，讓我們討論一些其他的系統，并定義一些聲音術語。



另外的一個系統是Miles聲音系統。Miles是一家公司，它為你的代碼生產插件，在充分利用每塊聲卡時處理所有必須的到特定聲音卡的說話（比如Sound Blaster Live！系列，或者老的A3D聲卡）。它非常像一個API前端，捆綁了一些額外的特征在里面。 在其他事物當中Miles讓你存取一些事物像MP3解壓縮。 它是很好的解決方案，但像任何事一樣，它花費金錢并是你的代碼和硬件之間的額外一層。雖然對於快速的聲音系統制造，它非常有用，而且他們有段時間了，因此他們的確精通自己的業務。



聲音術語
讓我們開始障礙和閉塞。它們聽起來一樣，但不是這樣。閉塞基本上意謂著一個聲音在播放時聽者在他們之間有一些閉合的障礙物。



比如說，在NOLF2的一個屏幕鏡頭上你聽到房子里面壞蛋的聲音。你能聽到他們，但是他們的聲音相當低沉而沙啞。障礙是相似的，但是你和聲音之間的障礙物并不是閉合的。一個好的例子就是在你和聲源之間有一根柱子。由于房間中的回聲你仍然聽得到這個聲音，但是它和聲音直接傳遞到你的耳朵里是不同的。當然這確實依賴于知道在你的耳朵和聲源之間的直線上是什么。而且根據房間的大小，聲源到你的距離等等，需要的處理能變得相當耗時。后面我們將會談到跟蹤--足可以說它時常是比較慢的幀速率的原因。Quake III 里面的A3D 代碼做了這些事情，關閉這些選項通常能夠提高幀速率。Tribe 2 是這種弊病的另外一個受害者。關閉3D聲音選項則你的幀速率立即好轉，這在你考慮Tribes世界有多大和你能看見多遠時有意義。



接著是聲音物質的特征。大部分聲卡可以讓你能夠用可定義的過濾器作用于聲音從而修正播放的聲音。例如，在水下，或者在一個布料遮蓋的房間中，或者在一個長的走廊中，或者在歌劇院，聽到的聲音有著很大的不同。能夠根據你所處的環境改變你聽到聲音的方式是相當不錯的。

我們回到Eagle… 這是一個編輯器，允許多數第一人稱射擊游戲地圖設計者將他們的地圖導入到這個工具，然后構造簡化的幾何形體來為實際游戲引擎中的EAX代碼產生一個聲音地圖。其思想是你不需要一個真實的圖形地圖的復雜幾何形體來模擬聲音環境。你也能夠給產生的簡化地圖分配聲音物質，這樣聲音環境就能夠動態地改變。我親眼目睹了這在Soldier of Fortune和Unreal Tournament上的示范，確實相當引人注目。 我這在財富和 Unreal 巡回賽和它的軍人上真的對示范是證人相當醒目. 當你跳入水中時，聽到所有的聲音改變，這是一個非常令人沉浸的經歷。

好, 讓我們繼續吧。

對于游戲機，由于靜態的硬件，你的各種可能性會更受限制 — 盡管在PlayStation 2和Xbox上，硬件相當不錯。我說的限制，僅僅是指擴展，而不是它所能夠做的。我一點也不會感到驚訝看到這些游戲機上的游戲很快支持杜比數字5.1（Dolby Digital 5.1）輸出。Xbox ，由于它的 MCP 音頻處理器，能夠將任何游戲音頻編碼為5.1，并且游戲不需要特別編碼就能利用這個特征。杜比（Dolby）把ProLogic II 帶到了 PS2 上，并與Factor 5合作為GameCube游戲實現了ProLogic II。在 Xbox 之上，Halo, Madden 2002 和 Project Gotham Racing等游戲都有5.1杜比數字音頻內容。DTS最近也為 PS2 游戲開發者發布了SDK，為這個平臺上的游戲帶來了降低了比特率的DTS音頻版本。



位置的聲音--一個復雜的世界
現在有一些很少有處理的聲音空間化問題。我說的是把聲音放在一個真實的3D世界中。有四個揚聲器在你周圍是一個很棒的開始，但這仍然只是在二維方向。在你的上方和下方沒有揚聲器，你沒有真正獲得3D聲音。有一些聲音調制過濾器試圖解決這個問題，但實際上沒有真實東西的代替物。當然真實地大多數游戲多半只是在二維方向上，因此這仍然不是太大的問題。

實際上任何聲音系統最重要的特征之一是把聲音混合在一起。根據你所處的位置，空間中聲音的位置，每個聲音的音量大小，一旦你決定了實際上你能夠聽到的聲音，然后你必須混合這些聲音。通常聲音卡自己處理這些，這首先是聲音卡存在的主要原因。然而，外面有一些引擎決定首先用軟件做一次‘預混合’。直到你著眼于一點點歷史以前，這并沒有真正地帶來多大的意義。



當聲音卡最初問世的時候，有許多不同的混合方法。一些聲卡可以混合8種聲音，一些單位16種，一些32種，等等。 如果你總想聽到16種可能的聲音，但你不知道聲音卡是否能夠處理，那么你回到了嘗試和試驗的道路上 — 就是你自己用軟件混合。這實際上是Quake III聲音系統的工作方式，但提一個問題:"Quake III是為A3D和Sound Blaster Live！聲卡世界發布的，這比以前更加標準化，為什么還這樣做？" 這是個好問題。實際上Quake III的聲音系統幾乎每行代碼都和Quake II中的聲音系統一樣。而且Quake I，甚至Doom也是這樣。你想一想，向上直到 A3D 聲卡和 SB Live! 聲卡，許多年來聲音系統的需求沒有真正地改變過。兩個揚聲器，二維方向，音量簡單地隨著距離減小。從Doom一直到Quake III沒有發生太大變化。而且在游戲行業中，如果不是迫不得已，別理會它。

通常你會僅僅使用DirectSound為你做聲音混合，因為它會可以使用的聲音硬件，或者轉而依靠軟件，很多地方就像DirectX為3D顯示卡所做的一樣。在 90% 的聲音情形中，依靠軟件混合對你的幀速率沒有真正發生太多不同。當DirectSound在一些狂熱的編碼者眼中甚至還不是一絲光線時，Doom引擎就已經產生了。它從來沒有得到更新過，因為它從來就沒有真的需要更新。

當然，你可以使用 SoundBlaster Live！聲卡的一些聰明特征，例如房間的回聲特性: 一塊石窟，或一個禮堂，一個巨穴, 一個足球體育館等。而且你真的應該使用由硬件提供的混合器，畢竟，那是它存在的目的。這種方法的一個不足之處是程序本身時常無法獲得混合結果，因為混合是在聲卡內部完成而不是在主存。如果由于某種原因你需要看到產生的音量，你是運氣不好。



Music Tracks in Games（游戲中的音軌）
我們沒有過多的談到游戲中的音樂生成。傳統的有兩種方法，一種是簡單的音樂 .wav 文件(或同等物)。它被預先制作做好，準備運行，和最小忙亂。然而，這些在內存和回放時間方面很昂貴。第二種方式用預設的樣本編碼MIDI音軌。這時常比較節省內存，但缺點是必須同時把一些聲音混合在一起，因而會把聲音通道用光。



動態音樂就是根據在游戲中目睹的行動改變你的音樂的能力，比如探險用慢節奏的音樂，戰斗用快節奏的音樂。預先制作的音樂的一個困難之處是要合拍，因此你可以從一段音樂漸弱到另一段音樂，這對于MIDI音軌比較容易。盡管時常你足夠快速地淡出，或者一段音樂在播放另一段音樂之前已經消失了，你能僥幸不被察覺。



在我們離開這個主題之前，順便說一下，值得一提的是存在一些公司專門為你的游戲創作特定意義的音樂。FatMan(www.fatman.com) 就是一家這樣的公司。音樂可能比其他別的東西更加容易外包，這是他們存在的方式。

最后，游戲現在的事情自然是MP3格式，允許巨大的11 ：1的聲音樣本壓縮，然而在送到聲音卡之前只花費CPU很少的時間解壓縮。當我在Rave Software工作時，在Star Trek Voyager: Elite Force 中，我們設法用MP3在一張CD上面完全支持三種語言，仍然為較多的圖形留有空間。主要地，我們 MP3 只用于非玩家角色（NPC）的語音，由于游戲的全部音頻效果MP3流和動態解壓縮超出了硬件的處理能力，雖然在將來這是肯定可能的。比較新的格式，如來自 Dolby 的 AAC 和來自微軟的WMA，以將近兩倍MP3的壓縮率提供了相等或者更高的音頻質量（實際上一半的比特率），可能應用到將來的游戲中。

以上是這一章節的內容，下面將是網絡和連線游戲環境的開發。