Archive for the ‘Qk’ Category

Squirrel Stack Tracking

Friday, November 8th, 2013

Squirrel native dev lives and dies by the stack, so here are some notes on the effect each function has on the stack.

NOTE: SQUserPointer’s are noted as void*’s (since that’s what they really are).

SquirrelStack

Virtual Machine


Stack 0

Stack ?? (depends on args: 0, -1, +1, or both)

Stack +1

Stack -1

Pops a value and sets it to the value found on the stack.

Compiler


If (successful) Stack +1 else Stack 0

Use sq_gettop(vm) to check if it was a success.

Stack Operations


Stack 0

Stack +1

Stack -1

Stack +N or -N

Stack -N

Object Creation and Handling


Stack +1

Notably sq_typeof() also returns values like OT_FLOAT, but specific info can be retrieved from the stack. DO NOT FORGET ABOUT THE STACK! POP IF UNUSED!

Stack ?? (+1, or optionally 0)

Stack -N and +1

Stack 0

SQR – shorthand for SQRESULT (for use with SQ_SUCCESS() and SQ_FAILURE()).

Stack -1 then +1 (effectively 0)

Stack -1

Member names are actual members of classes. For example, “x” in a vec2 class. sq_setbyhandle and sq_getbyhandle are used to read/write data to class members referenced by the Member Handle. Complicated yes.

Calls


Stack ?? (-N arguments, 0 or +1 returns)

Stack 0

Stack +1

Stack 0 or +1

Stack -1

Object Manipulation


Stack -1

Stack -2

TODO: Push value first, then key?

Stack -3

TODO: Push value first, then key?

Stack ?? (-1 or 0 if return)

Stack ??

Stack -1 then +1 (effectively 0)

Stack 0

Stack +1

Raw Object Handling


Stack 0

Stack +1

Revised thoughts on Squirrel Math

Friday, November 8th, 2013

Been thinking a bunch about the vector math classes mentioned in the previous post (vec2, vec3, etc). I was ready to try proposing “something” to let you add .anything to a class to access members hidden in (say) userdata. As it turns out, that already works.

The _get and _set metamethods talk about indexes in the documentation, so I mistakenly assumed they were for creating array-like syntax (maybe they are too), but in actuality they do exactly what I want: let you handle .anything

This is, of course, a Squirrel implementation.

A better use would be to write these metamethods in native code. Also instead of a class, use a UserData type to hold the true data (vector parts, a matrix, a quaternion), and attach all functionality using a delegate.

Using native has the added advantage when it comes to the operators (metamethods). Types can be checked far quicker this way (as constant values like OT_FLOAT), so my concerns about wasted time doing checks for each type shouldn’t be much of a problem anymore. I have to give somewhere, and the flexibility Squirrel provides is worthwhile.

Creating instances though is the question.

We could have a delegate called “vec2_delegate”, and a native function called “vec2” (like how JavaScript classes work). Have the function push the UserData structure on to the stack equal to the values passed, then attach the vec2_delegate to it. The delegate has a _typeof method that returns “vec2”. Finally, so long as it’s the only new thing on the stack, the vec2 function say it returned a value, and thus will be assigned by reference to

Copy Constructor implementation will be native, detected the same way as before, but no longer a cloning issue as it’ll be native code.

Just a few things to figure out:

1. How to “throw null” natively (required by _get and _set).

2. How to differentiate between UserData types (vec2, vec3, mat4, etc).

3. How to create and popluate Userdata.

Unrelated, but sq_getmemberhandle(v,idx,&handle) looks useful for optimizing data read/writes between the VM and Native code. sq_getbyhandle and sq_setbyhandle. The SQMEMBERHANDLE type however only contains a bool (_static) and an index (_index), so I’m not entirely sure how we get quick-access to data yet. It looks like you may push the value you want to set (sq_pushint, sq_pushstring, etc) and then follow it with a call to sq_setbyhandle. Reading the information though, it sounds like Member Handles may be a class-only feature, so this may not be what I’m looking for.

Classes can have UserData associated with them! sq_setclassudsize sets the size of the UserData attached to a class. sq_setinstanceup and sq_getinstanceup are a pair of functions for manipulating a UserPointer associated with a class instance (not UserData). That said, calling sq_setclassudsize will automatically set the internal classes UserPointer to the location of allocated data. Following up with a call to sq_getinstanceup will tell you where to put your data.

What’s missing?

The only thing missing is a nice way of handling Floats with new Vector and Matrix types (as userdata).

One option is to have a “scalar” or “real” type that exists for doing math with Vectors and Matrices. MyVec.x *= 10 is going to work fine already, but MyVec *= 10 will not. I could put in similar code as before, a check “if ( vs.type == OT_FLOAT )” then treat it as a scalar, but that doesn’t handle the front case (MyVec = 10 * MyVec). That’s why I’m suggesting a .toscalar() or .toreal() function. toscalar() I believe makes the most sense, as the operations being performed are scalar math ones. In addition, the float can have a .tovec3() or similar to create boring (1,0,0) type conversions.

Vectors will already support using any float as arguments “vec3(0,MyFloat,12)”. So classes like 2×2 or 4×4 matrix should support constructing with equivalent vectors “mat2(vec2(1,0),vec2(0,1))”. If feeling very adventurous, take any combination of float and vector types.

Yeah, the only hold-out is the “NewVec = 10 * OldVec” case.

I don’t really want to disturb the standard Float type by introducing extra check “is the previous variable a Matrix?”. Requiring conversion via .toscalar() may not be unreasonable after all, even though all operations like magnitude and normalize are available inside the Float (well, my modified float anyway).

Squirrel Class Notes

Friday, November 8th, 2013

Squirrel features both Delegates and Classes for creating types and providing default values and actions. They are mutually exclusive, meaning you either use a class or a delegate (attached to a table/array).

The following is a collection of notes on Classes.

Sample Class

The class below features a constructor, metamethods (like operator overloading), and a few additional functions.

Create instances with function syntax

Creating an instance is a lot like JavaScript.

The constructor is called with mentioned arguments.

Not to be confused with the _call metamethod.

Problems to consider: Functions are not overloaded, but replaced

In C++ and GLSL/HLSL, it’s common to have operators overloaded that let you Add, Subtract, and Multiply vectors by other vectors, scalars and matrices. This can (mostly) be done in squirrel by looking at the type of the argument received by your metamethods. Here’s an example that supports scalar multiplication as well as vector multiplication (component wise).

The problem though is this is one way. I can do “MyVec * 10”, but “10 * MyVec” will not work. To make it work, you would have to add a custom _mul metamethod to the number type… except I’m not sure that’s allowed by Squirrel (custom delegates are, but metamethods?).

One solution would be to create a “real” type (similar to GEL), but Squirrel doesn’t exactly provide a nice way to create custom automatic type conversions, so the syntax wouldn’t be ideal (i.e. real(10) * vec2(10,20) everywhere). I do live with this in GEL, but it would be a shame to not have a cleaner syntax here. This also assumes custom math metamethods are not supported by Squirrel (and technically I have the source, so it’s not like I couldn’t add any feature I wanted).

It’s also a shame the operator code ends up being so complex. A function that started as one line (the last line) has become 5 lines to add float and integer support. Add in Matrix multiplication too, and then I start getting scared.

An alternative solution would be to add .tovec3() and .toscalar3() functions to the default float delegate (where tovec3() is (10,0,0) and toscalar3 is (10,10,10)). This would be a cleaner option, but is somewhat wasteful, especially once we get to higher order maths like Matrix multiplication.

One more option would be to add language level vector and matrix support. After all, a table with x,y,z slots can’t exactly be the most efficient thing. This would certainly be the most difficult though.

It’s a shame Squirrel doesn’t have a way of adding shadow members. Like a MyObject.x() without the brackets that could be assigned (MyObject.x = 10).

I shouldn’t be expecting a scripting language to be ideal for high performance math anyway. Like I’ve said, specialization is the key. That is what scripting is for. Leave the low-level to the low-level.

Copy Constructors: meh

This is inconclusive currently. I may have found a bug with what should be the ideal way of doing it. I’ve since posted a question to the forum on it.

What I would like to do:

What I have to do instead.

This works, but isn’t as elegant as cloning.

Weak References

All built-in types have a .weakref() function in their delegate (classes, instances, generators, tables, etc). Bools, floats and integers return the actual type, but everything else returns a weakref type.

The weakref type is exactly that, a type containing a reference. To access the data referenced, you can call the .ref() function.

If the data referenced runs out of references, it is recycled. Any weakrefs pointing to it will thereafter return null.

In addition, Squirrel does some cleverness with weakref types. If you omit the .ref() call, it will still return the data referenced. For the most part, a weakref will act like the real type, except typeof will be weakref instead of the original type. To get the type referenced, you do typeof on the value returned by .ref().

Strong References

Okay, the reason for the brief exploration of weak references was because of an experiment I was doing with strong references.

Contrary to C++, instead of an assignment operator being called, a strong reference is created. Thus both Pos and Old point to the same data. In this way, Squirrel works like Java/JavaScript/C#, in that everything (other than int/float/bool) are references.

Instead, one should use clone to make a copy.

Clone performs a shallow copy, meaning all top-level members are copied. Integers, Floats, and Bools work as expected, but most other types are references, which may not be the behavior you desire.

Cloning Classes

In the case of a class, you can write a simple _cloned metamethod to handle the above case..

If a table, you can use a delegate like the following to automatically clone all members.

No General Purpose Copier for Classes

The general purpose copier doesn’t seem to work very well with classes. At least when I was trying to do it, I couldn’t get it working properly. Here’s a snapshot.

_get metamethod: are you even useful?

I have this code in my 3D vector class.

It’ll let you syntactically use the class like an array, but it’s not an array. So this is fine for inside Squirrel code, but if you have native code expecting an array, these will be useless.

Qk’s Squirrel “Reloading Scope” Guide

Wednesday, June 12th, 2013

This is a preliminary document. A first pass, a draft, talking through implementation notes from a piece of a project with the working name “Qk”. It might be something cool. I’m not telling. ;D

The programming language seen below is squirrel, but this specific nuance is something in my engine.


Reloading Scope

“Reloading Scope” is another name for the Global Scope of a script file. Take the follow code:

The variable Pos is in the global scope. Any and all code found in the global scope will be executed every time a script file is changed (automatically reloaded). So every single time this file reloads, Pos will reset back to [0,0]. This may not be the desired effect, as you may want the position to stay the same across changes.

To remedy this, simply move the code in to function scope.

REMEMBER: Both of these snippets effectively do the same thing, creating a global variable called Pos (in the root table). The only difference is the idea of “Reloading Scope” presented above.

Constants in Reloading Scope

For the most part, constants should be placed in Global Scope. This way, if you decide to change their meaning, the changes will affect all future references to the variable.

“Hard-coded” Data and Delagates in Reloading Scope

When code is dynamically reloaded, “hard-coded” isn’t really that “hard-coded” anymore. 😀

Delegates should be set immediately after the data is populated INSIDE the global scope. This way, during subsequent reloads when it trashes the old data, all your standard functions and default values continue to work correctly.

File Dependencies in Reloading Scope

Understanding Reloading Scope is important due to the way automatic dependencies work. All files that depend on this file will be flagged and reloaded at the same time. So changing this file may inadvertently cause another file to reload.

Each file is reloaded at max 1 time per refresh in Usage Order.

Usage Order is the order in which the files were referenced by each-other, starting with the root file (main.nut). Issuing a “requires()” command DOES NOT load a file, but instead queues it to load immediately after the current list of files have finished loading.

This is fine for code written for Function Scopes, but can be troublesome for Reloading Scope. After all, the newest version of a constant or class defined in a later file will be inaccessible/incorrect until all files have been loaded.

To work around this, push a function on to OnReload array. All functions in the OnReload array will be called immediately after all files have finished loading due to a refresh.

TODO: Consider priority model, giving functions inside OnReload a weight to control who gets called first/last. i.e. qkOnReloadCall( function(){}, Weight ); // Weight is optional. Default 0, highest first //

TODO: Consider “include()” function too which, if supported by squirrel compiler and VM, recursively compiles and executes code files on demand before finishing other ones. Need an #ifndef #define #endif construct, or #pragma once like function (global array of parent files, updated according to depth). Discourage the usage of this, but support it for hackability. “include()” should be thought of and used like a macro. “dofile()”. Overload sqstdlib dofile()?