A Musical Interlude

It's been waaay too long since I posted something on this blog that wasn't undub related, but over the past couple of days, I've actually been working on something pretty cool that I wanted to share here. It's a little app called Microtone. For anyone not familiar with microtonal music, basically the idea is to compose music using a scale other than the traditional 12-step scale used by almost all modern music. For some examples of microtonal music, check here, here, and here.

While microtonal music is certainly nothing new, very few instruments and tools exist for making it. Consequently, not that many people attempt to create music using non-traditional scales, and very little good music exists outside the boundaries of the standard 12 tones. I created Microtone as a quick app that people could use to experiment with different scales lengths, so that they might get ideas they wanted to use in serious compositions. Microtone itself is not really suitable for anything beyond experimentation, but I still think that it may serve the purpose of getting people interested in microtonal music. Or you can just mash your hands into the keyboard to make funny noises... Either way. ;)

Microtone works best with either Firefox, or a Chromium based browser (Chrome, Safari, Opera, etc.). Internet Explorer doesn't work at all (not even the newer versions), which is probably a surprise to no one. Using Firefox is preferable, because it appears to have a better audio mixer. Chromium based browsers may have weird sound distortion issues when playing more than two notes at a time. Also, if you have issues hitting multiple notes at the same time on your keyboard, you may want to look into a keyboard with n-key rollover. Anyway, give it a try and let me know what you think in the comments. ;)

Better Than Nothing

After much delay and procrastination, I am proud to finally present... a not yet complete undub of Grandia II. I'm sorry about this guys, I promised on many occasions that I would have this done by the end of the year, but this just wasn't a good time for me, and nothing has really gotten done in months. I don't want to make excuses, but the truth is I've been having a lot of difficulty in my personal life, the least of which is that my finances have been severely depleted by unexpected expenses, and it's very difficult for me to justify spending time on anything that doesn't have at least the potential to make money. I have added a Paypal donate button the to upper right corner of this blog in case anyone wants to help out. I'm not really asking for money in return for completing the patch or anything, but I'm not going to lie, this is likely not going to get done until my financial situation improves pretty dramatically...

Well, enough about that. I'd like to take a little time to talk about the background of this project and just exactly why it has taken so long for an undub of this game to appear. Grandia II on the Dreamcast was my first experience with the Grandia series. I was a die hard fan of the Dreamcast while I was in my teens, and as an avid RPG fan, buying Grandia II when it came out was pretty much a given. I'm not going to say the English voices ruined my experience, because they aren't the worst I've ever heard by any means, and Grandia II was the first RPG I played that even had voice acting so I didn't have much to compare it with. But the fact remains, when you listen to the Japanese voices of Grandia II, it's obvious which voices you would want to play the game with if you had a choice.

As undubs began to appear on the internet, I began to take notice. The first undub I can recall playing was Xenegears for the Playstation, and in my opinion it massively improved the experience of playing the game. I am not really fundamentally opposed to English voice acting or anything, it's just that most attempts to dub Japanese RPGS go horribly, because either the voice actors phone it in, or the lines they are given to read just sound unnatural in English no matter what you do. I actually preferred to play the Kingdom Hearts games in English because the English voices were much closer to those used in the movies the game was based on, so it's not like I never enjoyed playing a dubbed JRPG. However, Grandia II always stuck out in my mind as a game I would really like to be able to play with the original Japanese voices. I decided that it was worth my time and effort to at least attempt to undub the game.

Unable to find any existing dumps of Grandia II in the usual places, I bought a copy of the Japanese version on eBay and dumped it myself. I was already aware that in some cases undubbing can be achieved simply by copying files between versions of a game. After some experimentation I discovered that you could get the cutscene voices to be in Japanese by copying the GR2.AFS file from the Japanese to English version. But there were a lot of problems. First of all the voices were not synced with the English text. The English text would typically go by more quickly than the Japanese voices, causing the end of sentences to be cut off. Additionally, the battle voices were still in Japanese, there were cutscenes that needed to be subtitled, and the English translation was somewhat liberal, making it not sync up well with what was actually said in Japanese. The battle voices weren't too difficult to fix, as they were just located in different files and worked by simply copying them over. The cutscenes subtitles were doable as well, because Sega's encoder for the SFD video file format had been floating around the internet for years. But editing any resyncing the English text to match the Japanese voices? That was going to be a problem. To give you an idea, here is the overall picture of what has to be done to get at the actual English text.

The Grandia II game engine stores most of it's data in map files, located in the "MAP" folder on the root of the disc. This folder contains a series of files, with names following the format of "(4-digit hexidecimal number).AFS". For example, "3E01.AFS". There is no external indication what data each file actually holds. All of the game's script is located within those files, in addition to textures, geometry, and other miscellaneous data for each map in the game. AFS files are actually package files, which contain other files (similar to a ZIP file). If you can extract one, it will leave you with a series of unnamed files that contain the data for one map. The first file will always be the one that contains the script (don't ask me why). However, it will be compressed in a proprietary format called L62C, a variation of the LZW compression scheme. If you can somehow manage to decompress the files, you will be left with a file that contains the script, but also contains much incomprehensible data necessary for the map to function. Additionally, the text itself will contain special control characters that control how it is displayed, at what speed, and even what is happening in the actual game while the text is being displayed. If you can figure all that out, you will still need to figure out how to modify the pointer tables within the file containing the text, because unless you do, it will cause the game to freeze if you attempt to load a script file that has had text inserted. Even if you can figure out how to to all of that, you still have to figure out how to recompress the file in the L62C format so the game will be able to load it, and then repackage it in it's original AFS file.

Yea, there's a reason no one has undubbed this game yet. On an undubbing difficulty scale of 1 to 10, 1 being a game where you only have to copy a file from the Japanese version for it to work, and 10 being a game that requires hacking close to the level of what would be required in a full translation, Grandia II is about a 9. And actually, because I did end up re-translating the parts of the game with spoken dialog, this is in actuality a partial retranslation. It was very lucky for me, that I ran across a set of tools programmed by an Italian hacker named Mat for the specific purpose of allowing the game to be translated into other languages, because without those tools this undub would have never happened. I have some skill as a hacker, but reverse engineering proprietary compression formats from scratch is a little beyond me. So special thanks to Mat, because without his work this project would not have been possible.

If you have the tools you need to edit the game's script and reinsert it into map files, that should be all you need to fix the text to sync up with the Japanese voices... or so I thought. But as I discovered there were still cases where inserting text caused problems, as was stated in the documentation for Mat's tools. I had initially thought it was just an issue that could be fixed by inserting or removing bytes from the end of script section I modified, since that seemed to fix the problem in most cases. But then I ran into a map file where this was not effective, and any changes to the script that modified it's size caused the game to hang no matter what I did. I eventually figured out that this could be fixed by modifying a certain control code in the script that contained a pointer to an arbitrary location. While exactly why they though this was a good idea eludes me, it was an easy fix once I figured out what was going on. It was the figuring out what was going on part that took me so long.

I thought after fixing that issue a completed undub would follow in a relatively short period of time, but things didn't quite work out that way. For one thing, I'd forgotten that the amount of spoken dialog increases notably toward the end of the game. That combined with personal issues that have kept my very occupied lately have delayed this project too long in my opinion, and I felt like the people who have been following it deserved something, even if I couldn't deliver what I'd originally promised. So, I am releasing this preview version of my undub for you all to enjoy. ;) I haven't tested this patch as much as I would have liked to, so don't be surprised if there are still bugs or other issues. Please report any problems/mistakes you may encounter in the comments. This will also help ensure a higher level of quality for the eventual final release. Additionally, if you would like to help me out and possibly motivate me to get the final version done a little sooner, please consider donating. Donating is entirely optional and I can't promise it will dramatically impact my time frame for finishing this project, but it would really help me out in general.

Okay, so without further ado, here is a link to the patch:

Grandia II Undub Patch 0.90

Still Unfinished Business

Okay, so here's the deal. Earlier this year I made a post indicating that my Grandia II undub project was stalled out due to a problem that was causing hangs after inserting text into a certain file. The file in question is the one that contains the script for Mirumu village. I was originally hoping to get some outside help in getting the issue solved, but that didn't happen. I wasn't even able to successfully contact the person I was trying to get help from, and to be completely honest I'm not even sure if they're still alive. So that was kind of a downer... but, as it turns out it didn't matter. I was able to solve the issue, and at this point I see no reason to believe I shouldn't be able to get this thing done without making compromises.

I'll make a post later explaining the technical details of what was going on, but for now I'd like to thank anyone following this project for being willing to put up with my perfectionism. The issue I was facing was not a critical one, and I could have produced a very respectable final product by simply working around it. Just not one I would have been satisfied with.

The fact it took so much time to solve was more a question of my not having enough time to work on this project recently than anything else. That, and I was operating under the assumption I'd be able to get help from someone who'd already solved the problem for quite a while until I finally gave up. But now I'm over the hump, and barring any unforeen issues, a release should be coming soon. I say "should", because given the amount of difficulty this game has given me so far (the text insertion hang is not the only issue I've had to deal with), I may yet run into other issues that could delay a release before I get done. The overall project completion is currently sitting at around 90%, but right now I'm extremely busy and my time to work on it is limited. I think a release before the end of the month is possible, but don't hold me to that. In the mean time, here is a quick video I made to show off undubbed battle voices:



Note that any graphical glitches you may notice are due to the emulator I'm using to test my modifications (nullDC). There are emulators that can play the game without nearly as many glitches, but I use nullDC because it has a few features that make testing easier.

One other thing. I just want to be clear on the fact that I am going to release this undub for the Dreamcast version of Grandia II, and only the Dreamcast version of Grandia II. The Dreamcast version is the original, and best version of the game. The PS2 port is a mess, and the PC port is defficient in at least a few areas (sound/battle glitches). That being said, if anyone wants to take on the project of porting my undub to one of the other version of Grandia II, I am willing to share my knowlege and resources. Just as long as you understand that my experience with either version is limited, and I know for a fact there would be unique challenges in porting my undub that I do not intend to solve for you.

No News isn't Good News

Sorry I'm a bit late writing this post, but there's really not a whole lot to say. Basically, the reason my undub of Grandia II is not yet complete, is because I ran into some problems with text insertion that are preventing me from making edits to certain files. For what it's worth, I did my best to try to reverse engineer a solution myself, but the behavior of the particular opcode in the game's script responsible for causing hangs when text is inserted has thusfar eluded me.

That's the bad news. The good news is, I have made an appeal to a higher power that I fully expect will be able to help me get past this issue. I just can't say for sure when. Worst case scenario, I will try to work around the text insertion problem by making some edits to increase the brevity of the script in files where inserting text causes problems. Not huge ones, but ones that will undoubtedly be undesirable in terms of translation accuracy. So in the mean time sit tight. This thing is gonna get done, one way or another.

The Waiting is the Hardest Part

To be honest, I haven't made much progress on this undub in the past month, as I've been busy with other things. This is a situation that will soon be corrected. But for now, I thought I'd at least post another video showing off some stuff that has gotten done.



Note the FMV subtitles, which thankfully were a lot easier to get in the game than I expected them to be. Grandia II barely even has any dialog in the FMV sequences to begin with, so I guess I could have ignored those scenes altogether. But as I hope you've realized by now, this is not going to be the kind of half-assed copy and paste audio file undub that you're probably used to.

Unfinished Business

My release of a Gurumin undub earlier this year was a nice little project to get my feet wet in the world of undubbing. But truth be told, there are games out there that are in much more dire need of a proper undub. One of those games is Grandia II.

Grandia II is a game I've wanted to play with the original Japanese voices since I first put it in my Dreamcast more than ten years ago. However, actually getting it done was another matter altogether. For one thing, there was no readily available way to get ahold of the original Japanese voice files when I first started looking into the project. I solved that problem by buying a copy of the Japanese version of the game and a Dreamcast Broad Band Adaptor so I could dump the files myself. The next roadblock was the lack of a setup to efficiently patch the game and test changes. My Ruby-ISO-9660 library and NullDC together allowed me to achieve an efficient setup. And as a bonus, it will also allow me to make my final patch function with a GDI image of the game. The last roadblock was getting the game's script patched to use the timings from the Japanese version. Failing that, the text wouldn't sync up with the script in the scenes with voice acting, and in many cases the voice samples wouldn't get a chance to play completely. This problem was solved for me by a hacker who wrote tools to unpack, decompress, dump text from, insert text into, recompress, and repack Grandia II's map files. A task that I'm sure was no easy feat. For what it's worth, I did manage to solve an issue he never figured out, which allowed me to fix the crashes that occurred when you increased the size of the text you were inserting. ;)

With all roadblocks removed, I was finally able to produce this:

Now because I'm actually taking the time to retranslate the text from the voice acted scenes with my limited abilities in Japanese, it's going to take a while to get a finished product. How long is hard to say, but I'm going to estimate I'll have the final patch done by some time between the end of the summer and the beginning of next year. If I rushed I might be able to get it done sooner, but I have a lot of other stuff on my plate, and there's only so much time I can devote to something like this. Anyway, if you've always wanted to see Grandia II undubbed as badly as I have, this is definitely something to look forward to.

Maybe You Can Stop Internet Piracy After All

Before I get started, I'd just like to say that I am in no way endorsing this plan, or in any way suggesting it is a good idea. The only reason I'm even bringing it up is that it seems like such an obvious solution to limit the impact of piracy on the internet, that I'm really not certain why no attempts have been made to jam it down our throats yet. Maybe we can thank the fundamental lack of understanding politicians and lobbyists have about how the internet works? Either way, I thought I'd just bring up the idea in case anyone can think of a reason why it wouldn't work that I can't.

The idea is simply this: Why not require a license to operate an internet server? Piracy on the internet these days is primarily accomplished in one of two ways. Either by uploading pirated content onto a file sharing website, or by transferring it directly via P2P. If you were to require anyone who operated a server on the internet to be licensed by the countries in which the server was connectible, the first would quickly become impractical, and the second would be downright impossible.

In the case of file sharing websites, any site found to be operating a service dealing in large amounts of pirated content would have their license revoked, or at least suspended until they cleaned up their act. It would be pretty difficult to operate under the radar, because you'd need initial approval of your site to get a license in the first place. And even if you actually did manage to get a website up and running that dealt in pirated content, odds are someone would report you pretty quickly. Even a site that was inadvertently enabling sharing of copyrighted content could have their license suspended until they fixed the loophole that allowed it. As a bonus side effect, it would make the issue of operating an international website much less of a legal hassle, because you could choose which countries you wanted your site accessible in by only applying for licenses in those countries. There could also be some sort of global license for website operators who still wanted to enable worldwide access by default.

In the case of P2P sharing, you would need a license for other people to connect to you. These would typically not be given out to people on a consumer class connection, so you would not be able to engage in P2P connections at all. The only computers your ISP would allow you to connect to would be licensed servers.

On a technical level, such a system would actually be relatively easy and cheap to implement. It wouldn't require an overhaul or redesign of any existing systems. All you would need to do would be to implement a national white-list of IP addresses. ISPs could then add filtering rules that would make it impossible to connect directly to any server that wasn't on the list. Frankly, considering how much effort China puts into censorship, I'm shocked they haven't thought of this already. White-lists are not exactly a new concept, and they are extremely effective in limiting access.

Such a system would of course, not be perfect. It would for example, still allow people who did own servers to use them to connect to darknets along with other server operators and share pirated content that way. It would probably be possible to detect and track this kind of activity with enough effort though. But, for all intents and purposes, it would put internet piracy out of the reach of the average person. As far as the copyright lobbyists are concerned, that should be good enough.

The biggest hurdle to actually implementing such a system would likely be political. You thought SOPA got people upset? An internet server license bill would almost certainly lead to biggest uprising of internet activism the world has ever seen. Still, when you live in a country with a two party system, and both parties support the bill, the political danger either of the parties would find themselves in after passing it would be limited. I could envision a circumstance where the bill was muscled through at the beginning of an election cycle, and was put into effect long enough before the next election that most people would realize the law didn't affect them in any way. Politically, it could work if the timing was right.

I don't want to see a law like this passed. We have a top-heavy system controlled by a select group of individuals to begin with. The internet seemed to finally be putting a dent in that system, and requiring all internet server operators to be licensed by the government would almost certainly put an end to that. But I wonder, would we be able to stop a bill like this? After all the protests were over and done with, and the law was signed, would the people ever have enough political power to overcome those who have a vested interests in limiting our power again? Things like this keep me up at night.