Archive for May, 2013

SquirrelyJS – Squirrel Programming Language in the browser

Saturday, May 11th, 2013

Phew! Okay, my little science experiment from today (i.e. why I needed a makefile): SquirrelyJS.

Shot01

SquirrelyJS is a Squirrel Programming Language compiler and VM running in the browser. So it’s a bunch of JavaScript code, a bunch of C/C++ code compiled using Emscripten, all wired up to a shitty web page.

It’s barebones at the moment. You can edit/change/replace the code in the left side, and hit the Compile buttons to recompile it. Then click the Run buttons to run it, outputting to the left side.

The only commands that have any control are “print” and “error”. Both print text. print is black, and error is red.

For more Squirrely programming ideas (and an explanation of the test program) go here:

/2013/05/06/squirrely-things-about-squirrel/

Ultimately, the reason I made this was to do some benchmarking, but I’m too tired today to do anymore (I nearly didn’t even blog about it once finished).

So ya, Squirrels.

Barebones Useful Generic Makefile

Saturday, May 11th, 2013

This seems to happen a lot for me. I’m testing/playing around with some code thing, and I set up some cheeseball compiling in a shell script. Then I get to a point and it’s just not enough, the compiling is taking too long, or so one. My major projects often have a nice makefile setup going, but my experiments, not so much. That’s because writing a makefile, despite being the same thing again and again for me, is tricky to get right.

So here’s a generic makefile I’d typically use. I’m posting it here so I can steal it as needed, but feel free to use it and hack away at it; That’s what makefiles are for. 🙂

This makefile is designed for simple GCC/G++ usage. It’s designed to put temporary/object files in an “obj/” folder, and the final binary in the “output/” folder. Also the generated file can be executed from the make command (via ‘make run’). If you have a specific set of command line arguments you want to pass to the program, specify them in the ARGS variable.

Lines that are too long should end with a “\” character, which continues on to the next line. Make sure there are no spaces after the “\”!!!

I’ve also included an optional common tweak I often do. Un-comment the SRC_PREFIX line to assume all code files and paths specified are in a “src/” folder.

State of the Investigation

Friday, May 10th, 2013

So I’ve done a few scatterbrained posts lately that talk about things. Most of them had rather open ended conclusions, that were followed by wildly out-of-left field follow up posts. Here’s me making sense of that.

Evaluating Middlewares: Unity does good 3D, bad at JS

Unity’s “JavaScript” isn’t JavaScript, it’s better named UnityScript. UnityScript is kinda terrible, essentially being just a different syntaxed less featured C#. But var is allowed, so uhhh… *shrug*.

I still don’t care much for C#, but I think Unity’s 3D workflow is excellent. Fantastic even. I have 3D projects I want to do, and when the time comes, I’ve convinced myself I should do them in Unity.

That’s 3D only though.

Haxe NME is cool, impressive and all, but it seems more a home for Adobe Flash refuges right now. Adobe, I know not what they’re thinking. I’m very confused by what they’re up to anymore, so I’m glad the Flash community has a way out. That said, I’m not particularly attached to the stricter typed Ecmascript language used by Flash/Haxe/Loom. I like JavaScript. I think JavaScript is great. So I don’t entirely feel right with Haxe NME. Highly recommended for Flash devs and people not me. 😉

JavaScript is cool

JavaScript *is* cool. I’ve been working with it more and more lately. I made a game with Derek Laufman of Halfbot using JavaScript for Ludum Dare 26.

Toom09

Time was short during Ludum Dare, but we came up with some neat ideas, so we’re redoing the game. A “Director’s Cut” if you will. It’s still taking place in the one room, but we’re planning to make both the room and the interactions you have inside it way more interesting.

JavaScript is cool, but I’m hitting a limit of HTML5 Canvas 2D now in the latest builds of TOOM. Chrome is fine, but Firefox has seen a significant framedrop after adding a few more Alpha operations. I will be reintroducing frame skipping in to the code, but this doesn’t bode well for the near term “best user experience” of TOOM in the broswer.

That said, I fully intend to finish the TOOM Director’s Cut as an HTML5+JavaScript game. It just may be, before its time, when it comes to Canvas 2D performance. Working in JavaScript has been great, and inspired some really great alternative-to-Unity workflow ideas. I’ll come back to this. First though, what I think of App Packagers for Desktop HTML5 apps.

Packaged HTML5 Apps (Tool): Abandoned

In January 2012, I ported the unreleased Beneath Darkness prototype to some HTML5 packager. The company behind the packager was running a contest, and my intention was to enter the game in that contest. I wasn’t able to finish enough in time though, so I never submitted. That was fine though, as the winning games were definitely better and more complete.

The thing is, I can’t remember what that packager or the company was called.

It doesn’t really matter though. Like the Chrome Store, apps built for this packager were only for their proprietary HTML5 app store. I seem to remember a blue acorn, which may have been their logo. What disappoints me is that packager ran the game perfect. No issues with sound, no issues disabling linear filtering on image scaling, no issues loading files. Yet today, running the same prototype in both AppJS and Node-Webkit fails to load the map file. I’d be fine if Node-Webkit had no issues loading TOOM using PreloadJS, but it does. I might have been able to get NOOK working in Node-Webkit without any changes, but long story short I’m not happy.

Google’s Chrome App Packager stuff sounds like they solve the key issue (XMLHTTPRequest’s locally are okay), but it’s proprietary, and again only available to apps intended for distribution in the Chrome Store.

Even if it’s something that would be best fixed inside PreloadJS itself (support file://), that’s fine, but that’s not working 100% right now. I still have to do all my testing via a mini webserver running locally. My intent was to build a generic editor for a TOOM related project using HTML5, but I’ve decided against it. Building a Tool in HTML5 only saves me time if it works… right now… today.

I’m very unhappy with the state of packaging HTML5 apps as Desktop Apps.

Renstalled Empscripten, Embracing WebGL

NOOK, BEARly SEASONed, and the unreleased Last Gun prototype were made using Emscripten (C++ and JS). TOOM and the unreleased Beneath Darkness prototype were written in pure JavaScript.

I’m glad I went back to pure JavaScript for TOOM though, because HOLY HELL I learned how REALLY GREAT some of the features of JavaScript are. You know JSON right? Well it’s even better in JavaScript. 🙂

/nice-efficiency-things-about-javascript/

But like the headline says, I’ve since reinstalled Emscripten. My post from earlier today details this process, which was notably different than from when I did it 1 year ago. Better… except for the build time (70 minutes to build LLVM+Clang… and I had to do it twice).

/emscripten-2-the-emscripting/

And like the above article mentions, there’s now the AsmJS spec, which is an effort that makes JavaScript code run at only half the speed of native, versus the 6x slower than native I was promised last year.

I’ve been sort-of against WebGL, ironically, being an OpenGL guy. Against is a harsh word. More like, ignoring it. After all, Microsoft will never add it to Internet Explorer… or will they? Yes if the leaked Windows Blue version of Internet Explorer is to be believed, WebGL support is there now, so therefor coming soon.

Good.

What WebGL also brings is some potential Canvas graphics performance improvements. Your render code will have to be ported over to WebGL (OpenGL ES2), but with a little bit of batching, things get nice and quick.

https://www.scirra.com/boosting-mobile-html5-game-performance-with-webgl

Oh and Unreal Engine 3 now runs in the browser, thanks to WebGL and Emscripten.

So alright. If Unreal Engine 3 can do it, surely a WebGL accelerated 2D engine can.

Now for the final piece.

Squirrel! Like JavaScript but Nuttier!

Between work on the TOOM Director’s Cut… actually the other way around, I’ve returned to my Squirrel research in a big way. As far as languages go, Squirrel is very much like JavaScript, but with everything on my JavaScript wishlist already implemented and working great (operator overloading, separate integer and float number types, 32bit, delegates woo woo, etc). Here’s some exploration:

/2013/05/06/squirrely-things-about-squirrel/

My only beef with the language is that I’m comfortable with using var in JavaScript, instead of local like Squirrel uses. That said, they’re not the same, and even var in JavaScript is actually a problem: Such a big problem, that the EcmaScript 6 spec is introducing a new keyword let which works how you’d expect var to work.

Anyways, my mini-project of the day was to get Squirrel building in the browser (using Emscripten), and to build a very basic HTML interface for invoking the compiler and seeing the results. I’m betting that, since Unreal Engine 3 can run in the browser, it’s not unreasonable of me to expect instant execution of small Squirrel scripts. If that’s the case, then if paired WebGL, it may be reasonable to expect 30-60fps performance of a C++ game driven by Squirrel scripts. I’m not exactly interested in developing web games in Squirrel, but I’m investigating whether if I commit to Squirrel as a native game logic language, I can still generate playable web versions of games.

Long story short, I want the advantages I discovered in building TOOM in JavaScript in my Native Game Deving, and I already have proof that they do work.

Regrettably, I haven’t yet finished this Squirrel building project yet. Getting Emscripten working again ate up most of my afternoon (long Clang compiles). Then there was the power outage, and this blog post, so I think I’ve done enough for today.

I may try just compiling Squirrel with Emscripten, then I’m calling it a night.

EDIT: Yep. So far working just fine with NodeJS.

C++11 Support Feature Complete in Clang and GCC soon!

Now this is some awesome news. Clang 3.3 and GCC 4.8.1 are expected to be fully compliant with C++11 in their upcoming next versions.

GCChttp://gcc.gnu.org/projects/cxx0x.html
Clanghttp://clang.llvm.org/cxx_status.html

There’s still the problem of compiler vendors being slow to upgrade their GCC and Clang versions to newer ones, but this milestone should be a good kick-start.

Something also rather neat, Clang is adding C++1y (C++14?) features already. One that caught my eye, BINARY LITERALS! So just like you do 0x10 for the hexadecimal number 16, you’ll be able to do 0b11010011 to write binary. 😀

I have a header file, named Binary.h, that’s literally filled with every 8-bit combination of b0 to b11111111. Looks like I’ll be able to retire it some day. 🙂

End of Report

This post grew rather large ‘eh? Alright then, I’m done.

Emscripten 2: The Emscripting (or setting it up again, a year later)

Friday, May 10th, 2013

So last year, Derek and I created a little game called Nook. Nook was an interesting game that was half JavaScript and half C++, running in the browser (as 100% JavaScript). Things were still new with Emscripten. At the time it had only recently reached the point where it could work with standard C++ libraries without issue. Performance claims were about 6x slower than native. Everything worked out great, so I did little writeup on it:

/nook-and-emscripten-a-technical-look/

There were some talks at GDC this year on Empscripten, and regrettably I missed those. And just last week Unreal Engine 3 was ported to the browser using Emscripten. Thanks to recent optimization work, and the new AsmJS spec, its seems Emscripten code is close to 2x slower than Native in AsmJS optimized browsers. And as always, a great place to hear about interesting new things with Emscripten is on the lead developers blog and twitter.

http://mozakai.blogspot.ca/ | @kripken


Now formalities are out of the way, I’ve given myself a new project: Squirrel VM in the browser.

This blog post wont cover “SquirrelyJS” (a nickname I just came up with for the project), but setting up Emscripten again.

Setting Up Emscripten

I’ve just recently installed a new hard drive in my laptop, so I need to reinstall Emscripten. I’ll be following the tutorial here:

https://github.com/kripken/emscripten/wiki/Tutorial

Starting with the downloads, I had to install GIT (I’m an SVN guy). I like TortoiseGIT, because I also like TortoiseSVN (and TortoiseHG). TortoiseGIT requires MSysGIT to work.

It’s worth knowing that the Emiscripten you get out of GIT is the full thing usable as-is. No compiling is required. Once everything is installed correctly (Clang, Python), and your paths are set up correctly, it will work.

Python 2.7 I already had installed in C:\Python27. Emscrpten tries to be clever and use a file python2.exe, which doesn’t exist by default. This can be fixed later, but you can create a symlink, or make a copy of python.exe as python2.exe in the python directory. It’s 27k, so an extra copy of python.exe wont hurt.

NOTE: Java is required to use the closure compiler (i.e. JavaScript optimizer). For some reason this isn’t mentioned in the setup instructions.

Building LLVM+Clang

I use MinGW with MSys, and while yes there is an “experimental” download on the Clang/LLVM page, I wasn’t able to get it working. So instead, I had to build it from source.

WARNING: DO NOT BUILD FROM SVN! EMSCRIPTEN IS ONLY COMPATIBLE WITH THE CURRENT STABLE RELEASE! (Also skip the test-suite)

I followed the instructions here, with one change:

http://clang.llvm.org/get_started.html

EDIT: USE AS REFERENCE ONLY! You should only ever use the stable sources. Uncompress clang in to the tools folder as suggested (renaming clang-3.2.src to just clang), and optionally in projects compiler-rt (renaming compiler-rt-3.2.src to just compiler-rt). Skip the test-suite.

My change I did “../llvm/configure --enable-optimized --disable-assertions“, since I wanted a nice fast release build of LLVM+Clang, not a slower debug build with Asserts.

I also had to disable my current install of Clang, as it was trying to use Clang to build Clang+LLVM, which was broken. Only a problem if you tried to use the experimental package they provided.

An unfortunate downside, I was unable to use “make -j 4“, which uses multiple cores to compile. This is the 2nd time in the years I’ve been using MinGW+Msys I’ve come across something that make’s -j hasn’t worked (I can’t remember what the other one was, but it was a few weeks ago). It deadlocked on compiling the 4th file, which is not a good way to finish building anything. 😉

Compiling in a single thread took me over an hour. 🙁

Once built, “make install” to put it in my /usr/local/bin.

Configuring your system for Emscripten

First things first, we’re going to need to set some environment variables. You can do that here:

Control Panel->System->Advanced System Settings->Environment Variables

The Path is often the most important variable to tweak. Add new paths separated by semicolons (;). I added:

;C:\Python27\;C:\Emscripten\

There are some optional environment variables you can add here too.

EMSCRIPTEN – Where emscripten is (e.g. “C:\Emscripten\”).
LLVM – Where Clang+LLVM is. (e.g. “c:\MinGW\\msys\\1.0\\local\\bin”).
PYTHON – Python executable. (i.e. “C:\Python27\python.exe”).

Browse to the Emscripten directory with MSys, and run the following.

python emcc

Because its the first run, this will create an Emscripten configuration file in your home folder. Mine was here:

C:\MinGW\msys\1.0\home\Mike\.emscripten

In this file you can edit/set the location of Python, Clang/LLVM, etc. You can also change it from using python2.exe to python.exe if you want.

IMPORTANT: You need to set your temp directory. I modified the TEMP_DIR line as follows:

After all, the rest of the lines in the file use this os.getenv function, so why not temp too?

Okay! That should be it.

Now if everything is set up correctly, you should be able to run clang and have it not crash.

You should also be able to run emcc without explicitly invoking python now.

Node is NodeJS, the tool for running JS files on the command line (without an .html page).

Evaluating JavaScript to Native-App Packagers

Tuesday, May 7th, 2013

EDIT: For reference I’m leaving this post as-is (except for the new ending), but it seems AppJS is a lot more painful to use after all. Ugh.


Long story short, I need to write a tool with a UI. I want it to be portable, just in case. And because JavaScript is “fine”, I’m investigating what packaging options there are.

I’m not doing a very scientific test. All I’m doing is throwing TOOM, an adventure game Derek Laufman and I made for Ludum Dare 26 at it. If it works, I’m happy. If it fails, I’m not.

TOOM uses SoundJS and PreloadJS. PreloadJS wont work if the browser is reading from the local file system (i.e. file:///blah.html). So developing TOOM, I’ve had to use a tiny webserver running locally (currently Mongoose, was MiniWeb).

Chrome Packaged Apps

http://developer.chrome.com/apps/about_apps.html

Despite the name, despite the wording, this doesn’t create standalone apps. It creates content for distribution by Google’s Chrome App Store only.

It’s also not officially live yet. You can make apps, test them, but can’t Publish. Beh.

Node-Webkit

https://github.com/rogerwang/node-webkit

A good simple solution, based on NodeJS and Chromium, regularly updated (Intel Sponsored), but fails the TOOM test. It starts loading, but dies at an unusual place (complaining about one of my source files missing… which is definitely there if it can start loading at all).

Adds about 54 MB to the total size (~23 MB zipped).

AppJS

http://appjs.org/

A fantastic solution, also based on NodeJS and Chromium, and so far the only thing to pass the TOOM test (no audio and slower, so clearly using an older version of Chromium). The very very saddening part however is that there’s only 1 guy on the project, and he’s too busy.

Also adds about 54 MB to the total size (~22 MB zipped).

Visual Studio 2012

http://www.microsoft.com/visualstudio/eng#downloads

Untested, but this option is Windows only, and there may be other caveats. Any code written for DiskIO and file manipulation will only work on Windows. No WebGL support. That said, the JS code may compile down to CLI bytecodes (dot net), so performance might be better. Also how an app is structured may not mirror how a web project is created (index.html and all files). Untested though.

Adobe Air

http://www.adobe.com/devnet/air/air-sdk-download.html

Nope. Created my descriptor file, modified it to use the HTML file, then ran it with adl (adl D:\Toom\application.xml). After fixing the one bug it reported, it just sits there… waiting. Running I assume, but there’s no action.

NOTE: Instructions for running and packaging I found inside “AIR SDK Readme.txt” in AirSDK.

TideSDK

http://www.tidesdk.org/

Formerly known as Appcelerator Titanium for Desktop.

Finally, another one that passes the TOOM test. TideSDK is based on Webkit, but a seemingly an older version of webkit. CSS Fonts did not work, the framerate was even slower than AppJS, and no audio. Still, it was interesting. Downside is they seem to be short on funds too; Their blog reminds me of PBS.

More choices, no good ones

http://stackoverflow.com/questions/11015811/html5-desktop-wrapper-framework

Conclusion

There’s no safe/good/active choice, but AppJS impressed me the most. As far as portability goes, I think what I want to be using is something that combines Chrome and NodeJS, but not Webkit. Plus it gives me a reason to learn NodeJS too. My Browser performance and debugging experience is just so much better with Chrome, even though I don’t use it as a primary browser (I use Firefox instead).

Regrettably AppJS right now today is a bit of a gamble, since it’s just one dude behind it, and he’s made it clear he doesn’t have the time. But it does exactly what I want, so *shrug*. There’s talk of a Kickstarter, and I hope this sees the light of day. Totally a worthwhile project.

PLOT TWIST!

Not a good one though.

Something I hadn’t really realized before was how AppJS is actually working. From what I can tell, it’s actually running a small web server, i.e. the NodeJS part. Your main JS script file is standalone and disconnected from what runs in the browser context… YET, you control maximizing/minimizing windows, the contents of menus, task bar appearance from the NodeJS side. Your app is standalaone, caged inside the browser context.

Now, there may be a way to pass data between the NodeJS side and Chromium side, similar to making web requests in general. It’s not documented though, so *sigh* not sure where to start.

Node-Webkit might actually be more of what I want, since it’s about introducing the Node features in to Webkit. That said, it failed the TOOM test.

Ah well, enough of this for now.

(more…)

Squirrely things about Squirrel

Monday, May 6th, 2013

More notes. This article contains a longer more comprehensive list of differences, but I want a more concise list for both my own reference, and for when I’m teaching.

Squirrel 3 documentation.

(more…)