HyperJS Episode 5 - The Prototype Strikes Back

Prototypes and this Are Just Awesome -- in JavaScript

Posts in the HyperJS Series

• HyperJS Episode 1 - The Phantom Project
• HyperJS Episode 2 - Attack of the Accessors
• HyperJS Episode 3 - Revenge of the Script
• HyperJS Episode 4 - A New Hope?
• HyperJS Episode 5 - The Prototype Strikes Back

Last time, I discussed the fourth step on the road to JavaScript in C# in my HyperJS Episode 4 - A New Hope post. It covers the creation of JSObject that returns undefined for members not yet defined, the skeleton implementation of String and Boolean classes as functions, including scripts using "using", and the Unit Tests that prove its working. All of the projects I talk about in this series are available on my GitHub repos: http://github.com/tchype.

In part 5, we follow the typical story arc and hit the dark second act. Now that I have some basics and a general framework prototype in place, let's see what happens with functions, prototypes, and this.

HyperJS Part 5: Functions Should Be Objects, Dammit!

Notice that in String and Boolean, toString and valueOf are both instance methods. I would like them to be attached to the prototype instead of recreated each instance. I then want to be able to change the impelementation or add more members to the prototype and see the existing objects take on that functionality. Let's see what that would take...

Prototype Instances

Even though I had solved delegating property access/method calls to parent objects with HyperHypo, there's something distinctly different about actual JavaScript's prototypes. Prototypes are actually references to the functions that are the constructors for the objects. In JavaScript, you can dynamically add properties and methods onto the constructor function object! These end up being similar to static objects on a type, but with a BIG difference: references to "this" inside of those functions actually resolve at execution time to the instance of the object that the constructor function created when it was called with new, even though it actually resides on the prototype (static)!

Here's a JavaScript example, based on the Person class I defined in my 3rd post. I had defined:

• private instance variables _firstName and _lastName
• public instance getter and setter variables getFirstName(), getLastName(), setFirstName(), and setLastName()
• and a toString() method that concatenated the _firstName and _lastName private variables together with a space in between.

There's really no reason we need to have the toString() method be an instance method on Person. In the example as coded, it does use the closure of the instance constructor to access the private variables, but we could create one static-like function that can be shared across all instances of Person:

<script type="text/javascript">
  function Person(firstName, lastName) {
    var _firstName = firstName;
    var _lastName = lastName;

    this.getFirstName = function() { return _firstName; };
    this.getLastName = function() { return _lastName; };
    this.setFirstName = function(value) { _firstName = value; };
    this.setLastName = function(value) { _lastName = value; };
  }
  // Outside the constructor function definition 
  Person.prototype.toString = function() { 
    return this.getFirstName() + " " + this.getLastName();
  };
  ...
  var me = new Person("Tony", "Heupel")
  var singer = new Person();
  singer.setFirstName("Paul");
  singer.setLastName("Hewson");
  alert(me.toString());     // Outputs "Tony Heupel"
  alert(singer); // toString() called automatically and outputs "Paul Hewson" (but should have made it say Bono, somehow)
</script>

Again, even though toString() is now only one function that is shared across all instances of Person, the magic of "this" allows it to act like an instance function that works against the public members of a Person instance and appear as if it were an instance function. Some key things about JavaScript prototypes:

1. Prototype itself is actually the "static member" in that it is a single object of type Object (unless you specify otherwise) that applies to all objects of that type
2. When you add members to prototype, you're actually adding them to an instance of an object that is assigned to prototype, and that then allows all instances of the object type it is attached to to appear to have that function--even instances of objects already created previously!
3. In addition to adding or changing existing members on a prototype of an object, you can also totally replace the prototype object instance of another class (by default, all have a prototype of type Object, but you can change that at any time to create a "subclass" of another object)
4. Again, the magic of "this" cannot be overstressed...

Even though HyperHypo supports a Prototype member, it just points to an instance of an object. This works fine in many cases, but what if you want your class to inherit from another class that is not JSObject? You would simply assign a new value to the Prototype member, right? Not quite...Prototype is an instance member and cannot be made static or all instances of JSObject would share the same prototype. And if you just change it on one instance, then the others do not see the new functionality either. So, I made three key changes:

1. JSObject has a JSTypeName: This is just a string that in regular class-oriented programming could be replaced with this.GetType().Name, but in this style just needs to be set when a new instance is created.
2. The JS instance has a Prototypes dictionary: To ensure that all new instances of a particular class share the same Prototype object, the first time an object of that type is created, it creates it's prototype and adds it to the global object's Prototypes dictionary with a key of the JSTypeName value; any subsequent creation of an object of that type will use the same Prototype object from the dictionary to ensure that we can add members to the prototype and all instances of that type get the functionality.
3. JSObject gets notified when a prototype for a type is assigned a new object instance: To inherit from something different, we could--on the fly--say something like JS.cs.Prototypes[JS.cs.String(null).JSTypeName] = someNewObjectType;. While this would change the base object for any new instances created, it doesn't update any old ones, since Prototype is a reference to an object, not a getter that looks at the global Prototypes dictionary. Rather than overriding HyperHypo's Prototype property getter and setter, I simply had every JSObject get notified when this changes and it sees if the change was to the prototype key for that object's JSTypeName.

How Well Does That Work?

I just-so-happen to have some unit tests that help prove what this looks like and how well it works:

[TestMethod]
public void PrototypeFun()
{
    dynamic s = JS.cs.NewString("hello");
    dynamic thing = new JSObject();

    thing.Prototype.bizbuzz = new Func<string, string>((name) => "hello, " + name);
    s.Prototype.foobar = new Func<bool>(() => true);  // Check it doesn't inadvertantly set ALL Prototypes from root object to have foobar

    // bizbuzz method available on all objects
    Assert.AreEqual("hello, tony", s.bizbuzz("tony"));
    Assert.AreEqual("hello, tony", thing.bizbuzz("tony"));

    // foobar set oon string prototype, but not exposed to object
    Assert.IsTrue(s.foobar()); 
    Assert.IsFalse(thing.foobar); // Feature detection -- not set on object prototype


    Assert.AreEqual(3, thing.Count); //foobar should not show up on JSObject's prototype....
    Assert.AreEqual(4, s.Count); // foobar and bizbuzz are available to previously created string instances
            
    // Create new string prototype and set it
    dynamic newPrototype = new JSObject();
    newPrototype.skunkWorks = "skunky!";
    s.SetPrototype(newPrototype);  // skunkWorks now available on string

    Assert.AreEqual(3, thing.Count);  // Updating string's prototype shouldn't mess with Object
    Assert.IsFalse(thing.skunkWorks); // Feature detection - make sure skunkWorks not on object
    Assert.AreEqual(4, s.Count); // toString, valueOf, skunkWorks, and bizbuzz (no foobar)
    Assert.AreEqual("skunky!", s.skunkWorks); // Can access it through string instance previously created
    Assert.IsFalse(s.foobar);  // Feature detection - since prototype was changed, no longer there!
}

As you can see in the tests above, you can add members to prototypes and have "sub objects" pick up the functionality, in very basic cases. It turns out that this approach fails in the most critical case: adding methods to the prototype that act as instance methods! The other cases that work properly are:

• You are adding a member property to the prototype
• You are adding a function that is actually a static function and does not require any knowlege of an instance (e.g., Date.getDate())

How to Implement Instance-Like Methods on Prototypes

How interesting that one of the things I thought was solved early-on in HyperDictionary is actually coming back to be a serious issue. Next, I went down the route of trying to figure out how to implement instance-like methods on the prototype for String and Boolean. In both cases, the only obvious thing to do was to create a static method that takes a "this" or "self" as the first argument and add it to the prototype, and then create instance methods that pass "self" into the prototype method. This works at first but is dumb. Most of these functions are small, so creating an instance wrapper and a prototype version is just a waste of space. Additionally, you still can't just add a method to the prototype that looks like an instance method, and adding new instance wrappers to existing object instances is dumb as well.

Sure, I could hook into my Prototype update mechanism, but that was already a hack around the fact that functions aren't objects and you can't attach properties to them. I even investigated trying to subclass a delegate (System.MulticastDelegate) in a JSObject, but I knew that would fail since you have to be a special language service/implementor in order to able to do anything to System.Delegate or System.MulticastDelegate. If only I had the flexible "this" available to me...

A Look at call() and apply()

One of the ways you can control the "this" scoping in JavaScript is to use the call() or apply() functions on a function object. They essentially let you call the function, but specify the "this" of said function:

// Assume Person has a saySomething method on its prototype:
Person.prototype.saySomething(wordsToSay) { 
  return this.getFirstName() + " said '" + wordsToSay.join(' ') + "'";
}

var p = new Person("Tony", "Heupel");
// The lines below is the same as caling p.saySomething("Hello", "There");
// and both return "Tony said, 'Hello There'"
Person.prototype.saySomething.call(p, "Hello", "There"); 
Person.prototype.saySomething.apply(p, ["Hello", "There"]);

Perhaps the answer lies somewhere in there. Maybe I could update or sublclass JSObject to make a JSFunction where it is a dynamic object that has a call() and/or apply() method that calls the function. But then how to make the syntax of calling any function (not just a wrapped constructor function) look like o.foobar("biz") reasonably flexibly and not require a ton of work on the object/function author to make it work.

Hello, Python?

Then, something suddenly struck me: oh, crap. This is all ALMOST EXACTLY how Python works! Python objects, including classes (yes, classes are objects themselves), instances of classes, and functions (yes, functions are first-class objects as well) all have a Dictionary of name/value pairs (object.__dict__) at each level (instance, class, metaclass, etc.), you can use indexers to set attribute names and then access them with dot-notation, functions are objects with a __call__ attribute and methods take self as the first argument, and so on...

The most disappointing thing: OK, I could do this, but then I'll need an interpretor/compiler in order to get the syntax to stay reasonably similar to real JavaScript, and I'm trying to avoid that like crazy. Additionally, Python already runs on every platform--including the .NET CLR/DLR--and has a full ecosystem and 20 years of work behind it. ARGH!

A Reasonable Prototype Outcome

When I started the actual HyperJS portion of this work, I was very specific in my goals that I thought would prove it to be a good idea or doomed from the outset. Heading down this path is definitely a possibility and I am proud that I set up my goals such that I could find this early--within 2 weeks of starting out on this crazy journey in my spare time.

I'm not personally willing to head down this path any further at this time. This is the dark "second act" of the HyperJS story. It may never emerge into the triumphant "third act" (Return of the HyperJS Jedi, or something); or maybe, because it's Open Source and on GitHub, someone will find it and want to keep going with it before I decide to. That would be awesome if this project has a real useful future that I'm holding up by not pursuing it for now...

In the meantime, I'll probably dive back into Python and Ruby (I found the Pickaxe book at Goodwill for $2, like new!) for established solutions, and Node.js and Jaxer for up-and-comer use of JavaScript on the server. That just seems like a better use of my time (and my wife will be happy that I'm finally done being "almost done" with all of this "spare time" investigation).

Thanks for going on this ride with me. I hope it was well worth the read and a good brain-stretcher and maybe plants seeds of ideas into your head. Remember, even though HyperJS has stalled, HyperCore still has some really cool stuff in it that I think is useful. If you find it useful and plan to use it, let me know!

HyperJS Episode 3 - Revenge of the Script

JavaScript Style Objects in C#

Posts in the HyperJS Series

• HyperJS Episode 1 - The Phantom Project
• HyperJS Episode 2 - Attack of the Accessors
• HyperJS Episode 3 - Revenge of the Script
• HyperJS Episode 4 - A New Hope?
• HyperJS Episode 5 - The Prototype Strikes Back

Last time, I discussed the second step on the road to JavaScript in C# in my HyperJS Episode 2 - Attack of the Accessors post. It covers the deficiencies of ExpandoObject and the simple dynamic class I created to allow JavaScript-style getter and setter notation in C# - HyperDynamo. All of the projects I'll talk about in this series are available on (my first) GitHub repos: http://github.com/tchype.

In part 3, the fun with crappy names continues. I'll be covering how combining the classes created in parts 1 and 2--along with closures in C#--allows for JavaScript-style Object declaration instead of class-based specification.

HyperJS Part 3: HyperHypo

Now that I have a HyperDynamo class that allows getters and setters to a dictionary via dynamic dot-notation syntax (foo.bar) and via indexer syntax (foo["bar"]), and I have a HyperDictionary class that allows for prototype-style inhertance and overrides of key/value pairs, let's combine them and see what kind of fun we can have.

DumpEnumerable

Here's a helper function I use often to display the members of a dynamic, enumerable class so we can see exactly what's defined where:

private static void DumpEnumerable(dynamic thing)
{
  foreach (object o in thing)
  {
    Console.WriteLine(o);
  }
}

Using HyperDynamo and HyperDictionary Manually

In part 2 of this series, I created a three-level set of HyperDictionaries: top, second, and third. Here's a quick synopsis:

• top: [eyes, brown], [hair, pointy], [things, { "first thing", "second thing" }]
• second (inherits from top): [hair, straight]
• third (inhertis from second): [things (extend), { 3, 4, 5 }], [hair (remove)]

Now, let's create a dynamic object (dynoThird) based on the third HyperDictionary and use it:

Console.WriteLine("Using HyperDynamo with HyperDictionary MemberProvider to show\ndynamic mappings and inheritance!\n==================================================");

dynamic dynoThird = new HyperDynamo(third);  //Manually use HyperDictionary with HyperDynamo
dynoThird.toes = "third toes set through dynamic property";
Console.WriteLine("eyes:\t{0}", dynoThird.eyes);
Console.WriteLine("toes:\t{0}", dynoThird["toes"]);

Console.WriteLine();
try
{
  //Should throw an exception since it got removed at this level
  Console.WriteLine("hair:\t{0}", dynoThird.hair);
}
catch (Microsoft.CSharp.RuntimeBinder.RuntimeBinderException rbe)
{
  Console.WriteLine("EXPECTED EXCEPTION SINCE PROPERTY WAS REMOVED:\n{0}", rbe);
}

Console.WriteLine("Properties in the HyperDynamo object built off of 3 levels of HyperDictionary\ninheritance and a couple dynamic property setters\n========================================================================");
DumpEnumerable(dynoThird);

Which outputs:

Using HyperDynamo with HyperDictionary MemberProvider to show
dynamic mappings and inheritance!
==================================================
eyes:   brown
toes:   third toes set through dynamic property

EXPECTED EXCEPTION SINCE PROPERTY WAS REMOVED:
Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: 'TonyHeupel.HyperCore.Hyp
erDynamo' does not contain a definition for 'hair'
   at CallSite.Target(Closure , CallSite , Object )
   at System.Dynamic.UpdateDelegates.UpdateAndExecute1[T0,TRet](CallSite site, T
0 arg0)
   at HyperActive.ConsoleApp.Program.CombiningHyperDictionaryWithHyperDynamo(Obj
ect third) in C:\Users\Tony\Documents\Visual Studio 2010\Projects\HyperActive\Hy
perActive.ConsoleApp\Program.cs:line 168

Properties in the HyperDynamo object built off of 3 levels of HyperDictionary
inheritance and a couple dynamic property setters
========================================================================
[things, System.Linq.Enumerable+<UnionIterator>d__88`1[System.Object]]
[toes, third toes set through dynamic property]
[eyes, brown]

HyperHypo - More Than C#, Less Than JavaScript

The naming suck-fest continues, but there is thought behind it regardless. I created a sublcass of HyperDynamo called HyperHypo with these two key characteristics:

1. Requires a HyperDictionary for the MembershipProvider
2. Has a Prototype property, where you can set the Prototype of the object, ala JavaScript, but the Prototype is another HyperHypo instance.

While building the HyperDictionary tree yourself is fine, it's kind of lame, so putting a class around it that takes care of the details was important (but I didn't want to REQUIRE that people use HyperHypo if they didn't need or want it). The second point is a key component: JavaScript is more Object oriented than Class oriented, so we are talking about actual instances of objects here, not class definitions. Basically, setting Prototype sets the inner HyperDictionary's parent, and that's it!

HyperHypo With Functions

There were some great additions to .NET 4 in the realm of dynamic capabilities, especially the Func<...> type. Basically, there's like 17 types of Func<> declared, where Func is a delegate that returns a type. In each of the definitions, the last type declaration is the return type and any ones previous are the types of the arguments (ordered). So for example, creating a function that takes no arguments and returns a string would be of type Func<string>. A function that takes an integer and returns a Foo object would be Func<int, Foo>. And so on...

It turns out, the "value" part of the key/value pair can be any object--including Func<>. So hold on, buckaroo: inheritance and functions and lambdas, oh my!

...
// SayHello function defined in same class...
public static string SayHello() { return "Hello"; }
...
Console.WriteLine("Using HyperDynamo with HyperDictionary membership provider\n==========================================================");

Console.WriteLine("First example: prototype inheritance where only one.Whassup is set");
dynamic one = new HyperHypo();
one.Whassup = new Func<string>(SayHello);
Console.WriteLine("one.Whassup(): {0}", one.Whassup());

//two inherits from one (set's it's prototype)
dynamic two = new HyperHypo(one);
two.HowsItGoing = new Func<string, string>(name => String.Format("How's it going, {0}?", name));
Console.WriteLine("two.Whassup(): {0}", two.Whassup());
Console.WriteLine("two.HowsItGoing(\"buddy\"): {0}", two.HowsItGoing("buddy"));

Outputs:

Using HyperDynamo with HyperDictionary membership provider
==========================================================
First example: prototype inheritance where only one.Whassup is set
one.Whassup(): Hello
two.Whassup(): Hello
two.HowsItGoing("buddy"): How's it going, buddy?

HyperHypo With Closures

First, a brief detour into JavaScript. Here's how you define an object with private properties in JavaScript (if you aren't familiar with this, there are plenty of books and web sites that describe how this works in detail):

<script type="text/javascript">
  function Person(firstName, lastName) {
    var _firstName = firstName;
    var _lastName = lastName;

    this.getFirstName = function() { return _firstName; };
    this.getLastName = function() { return _lastName; };
    this.setFirstName = function(value) { _firstName = value; };
    this.setLastName = function(value) { _lastName = value; };
    this.toString = function() { return _firstName + " " + _lastName; }
  } 
  ...
  var me = new Person("Tony", "Heupel")
  var singer = new Person();
  singer.setFirstName("Paul");
  singer.setLastName("Hewson");
  alert(me);     // Outputs "Tony Heupel"
  alert(singer); // Outputs "Paul Hewson" (but should have made it say Bono, somehow)
</script>

A couple keys:

1. Variables with function scope (those declared using "var") are private variables and retain their state because of the closure of the function
2. The public properties/methods on the Person class instance are created by adding them to "this"
3. When using the "new" keyword, you get a new object, which is an instance of the Person() function (the "this" inside the function) with it's closure intact, so each copy has it's own set of variables.

So let's do this with HyperHypo in C#!

class Program
{
  static void Main(string[] args)
  {
    Console.WriteLine("Using HyperHypo (HyperDynamo with HyperDictionary)\nand closures to create JavaScript-style object declarations\n==========================================================================");
    // Define the class as a function constructor and 
    // private variables using closures (inline, without a separate class!)
    dynamic Person = new Func<string, string, dynamic>(delegate(string firstName, string lastName)
    {
      var _firstName = firstName;
      var _lastName = lastName;


      dynamic p = new HyperHypo();
      p.getFirstName = new Func<dynamic>(delegate() { return _firstName; });
      p.getLastName = new Func<dynamic>(delegate() { return _lastName; });

      p.setFirstName = new Func<string, object>(value => _firstName = value);
      p.setLastName = new Func<string, object>(value => _lastName = value);
                
      p.toString = new Func<string>(delegate() { return String.Format("{0} {1}", _firstName, _lastName); });
      return p;
    });

    dynamic me = Person("Tony", "Heupel");
    dynamic singer = Person(null, null);

    singer.setFirstName("Paul");
    singer.setLastName("Hewson");

    // Now output stuff and make sure the closures and functions work!
    Console.WriteLine("me.getFirstName():\t{0}", me.getFirstName());
    Console.WriteLine("me.getLastName():\t{0}", me.getLastName());
    Console.WriteLine("singer.getFirstName():\t{0}", singer.getFirstName());
    Console.WriteLine("singer.getLastName():\t{0}", singer.getLastName());
    Console.WriteLine("me:\t{0}", me.toString());
    Console.WriteLine("singer:\t{0}", singer.toString());
    Console.WriteLine();

    // Notice that with the closure at the time the constructor was called,
    // each Person has it's own variable scope (closure) that does not
    // interfere -- even if the constructor is a static method!
    DumpEnumerable(me);
    DumpEnumerable(singer);
  }
}

Which outputs:

Using HyperHypo (HyperDynamo with HyperDictionary)
and closures to create JavaScript-style object declarations
==========================================================================
me.getFirstName():      Tony
me.getLastName():       Heupel
singer.getFirstName():  Paul
singer.getLastName():   Hewson
me:     Tony Heupel
singer: Paul Hewson

[getFirstName, System.Func`1[System.Object]]
[getLastName, System.Func`1[System.Object]]
[setFirstName, System.Func`2[System.String,System.Object]]
[setLastName, System.Func`2[System.String,System.Object]]
[toString, System.Func`1[System.String]]
[getFirstName, System.Func`1[System.Object]]
[getLastName, System.Func`1[System.Object]]
[setFirstName, System.Func`2[System.String,System.Object]]
[setLastName, System.Func`2[System.String,System.Object]]
[toString, System.Func`1[System.String]]

Holy Crap Balls, Batman!

OK...so that actually works! Certainly, the static nature of C# adds some funky limitations at first glance, but this is pretty freaking sweet! I've always been a little jealous of the Node.js crowd (mainly because I haven't had time to try to it out on my Mac yet) getting to do server-side JavaScript with awesome, non-blockingness, but this really made me think that some form of something resembling JavaScript in C# is possible!

Up Next - HyperJS Project

Well, now that we can create objects dynamically in a way very similar to JavaScript, it only seems like a good idea to see what the limitations are. What better way, I think, than to see if I can actually create JavaScript in C#, including Object, Boolean, valueOf, toString, Image, etc.. That should quickly tell me how off my rocker I am.

Again, why? Mainly, "because I think I can." Really, I should look at using F# Power Tools with their Lexer/Parser to just make a compiler for JavaScript that runs on .NET or something. But really, wouldn't that still just generate C# code anyway?

Every class covered to date is located in my HyperCore project on GitHub. The idea is that these are small and potentially distinctly usable classes for different purposes. I really see any venture into HyperJS or JavaScript type work itself a separate concern that builds on top of HyperCore.

In my fourth post in this series, I'll cover the (VERY early) attempts at creating HyperJS (or JS.cs) and the significant decisions to make and issues to overcome. Mixing dynamic with statics, singletons, extension methods for seamless integration of libraries, and all kinds of weirdness that has basically been fun but bizarre. Stay tuned!

HyperJS Episode 2 - Attack of the Accessors

JavaScript in C# IS NOT ExpandoObject

Posts in the HyperJS Series

• HyperJS Episode 1 - The Phantom Project
• HyperJS Episode 2 - Attack of the Accessors
• HyperJS Episode 3 - Revenge of the Script
• HyperJS Episode 4 - A New Hope?
• HyperJS Episode 5 - The Prototype Strikes Back

Last time, I discussed the first step on the road to JavaScript in C# in my HyperJS Episode 1 - The Phantom Project post. It covers the original intent of my weekend prototyping and the result: HyperDictionary - An implementation of IDictionary<string, object> with "inheritance," overrides, and extensions. All of the projects I'll talk about in this series are available on (my first) GitHub repos: http://github.com/tchype.

In part 2, I'll be covering how Rob Conery's post that I'll incorrectly label "Trying to implement Ruby Hashes In C# But C# Leaves A Lot To Be Desired In That Regard (I know, Oren, Use a Dynamic Language If I want Dynamism)" was an influence once I realized I was headed down this path, how I really just wanted a simple feature, and how I had to build it myself. Then, once I did it (very easily), I started down a weird and fun path of seeing if I could make JavaScript in C#.

HyperJS Part 2: HyperDynamo

Now that I had created a HyperDictionary, I was wondering how hard it would be to allow a Dictionary to be populated by a data source using the key/value pairs defined in the data store, but then accessed via the dot-notation of a property name. The thought here is that although a data-store-driven key/value pair allows you to arbitrarily add a key and value, typically you have to write code to use it anyway. With recordsets and dictionaries, you typically create constants somewhere in your code (because, after all, we all want to keep it DRY, right?) and then have this kind of annoying code all over your project:

public const FIRST_NAME_KEY = "first_name";
public const LAST_NAME_KEY = "last_name";
...

// Read from DB into an object
var user = new Person()
 { 
   FirstName = record[FIRST_NAME_KEY], 
   LastName = record[LAST_NAME_KEY],
   // ... more properties ... 
};
...

//Set the user's first name into a ViewData key
ViewData["user_first_name"] = user.FirstName;

Just to be clear: I don't actually code directly against the recordset in the view or controller...I like ViewModels and DTO objects and exception handling; this is just an obscene example that illustrates the point of using constants to reference field and key names.

After seeing Rob Conery's post on doing Ruby-esque things in C# (and having done some in MVC and side Rails projects), I wondered if it would be possible to do something more like JavaScript...wouldn't it be nice if you had the option to easily populate an object's properties by looping through the key names and setting their values (ignoring the tight coupling and issues with valid property name validation):

dynamic user = new ExpandoObject()
foreach (string key in myDictionary.Keys)
{
  user[key] = myDictionary[key];  //Actually, this is an Exception...
}

Even when we use constants for key names, we are essentially compiling the key names in for those keys we know and care about and write code against. And then, in cases of quick internal apps, prototypes, or just for eliminating the need for the stupid constants, we could just reference the properties we care about against the dynamic binding:

ViewData["user_first_name"] = user.first_name

ExpandoObject Is Not Good Enough

Although ExpandoObject is really cool, it relies purely on dot-notation property setters and getters (no indexers), so this doesn't actually serve the purpose. So I had seen the idea to back a dynamic object with a Dictionary in an MSDN article describing the DynamicObject class in .NET 4. The purpose of that was to demonstrate how you could change the behavior of the property name routing through TryGetMember/TrySetMember (in this case, to all lower-case); I'm sure this is what the IronRuby team used some of to map into ruby-style method and property names...but I digress.

Happy Birthday, HyperDynamo!

I created my own class, inherting from DynamicObject, backed by any implementation of IDictionary<string, object> that overrides the TryGetMember and TrySetMember and maps property names to key names directly and stores the value in the value portion of the KeyValuePair. I suck at naming, so Hyper ("more than", and also my Ayende-esque "Rhino-type" prefix) and Dynamo (for "dynamic") because it's more than the basic DynamicObject class. I also implemented the indexer on a string name and IEnumerable<KeyValuePair<string, object>> so that I could access properties via:

1. Dot-notation
2. Indexer (e.g., foo["bar"]) for both setters and getters
3. And use foreach to iterate through the properties

It's Your Birthday, Go HyperDynamo!

So, with ease, I can now do JavaScript-style assignment, retrieval, and iteration on HyperDynamo objects in C#! Take a look:

dynamic person = new HyperDynamo();
person.FirstName = "Tony";
person.LastName = "Heupel";
person["MiddleInitial"] = "C";

Console.WriteLine("Hello, {0}, {1}., {2}!", person["FirstName"],
                              person.MiddleInitial, person.LastName);

This displays what you would expect:

Hello, Tony C. Heupel!

That's Cute, But Hardly JavaScript

Very true. Again, my original goal was not to create JavaScript in C#, but rather to enable some JavaScript (and other dynamic language) style for reducing complexity of mundane work and impedance mismatch. But, did you notice something? While HyperDynamo uses a Dictionary<string, object> by default, it really only requires any object implementing IDictionary<string, object>--including a HyperDictionary with Prototype-style inheritance!

Next Up -- Part 3: HyperHypo

The sucky names continue, but things get interesting once you start combining what I have placed into my HyperCore assembly together, mix in a pinch of closures, and top it off with a smidge of real JavaScript concepts...check out Hyper JS Episode 3 - Revenge of the Script!

HyperJS Episode 1 - The Phantom Project

JavaScript in C# starts with Prototype Inheritance

Posts in the HyperJS Series

• HyperJS Episode 1 - The Phantom Project
• HyperJS Episode 2 - Attack of the Accessors
• HyperJS Episode 3 - Revenge of the Script
• HyperJS Episode 4 - A New Hope?
• HyperJS Episode 5 - The Prototype Strikes Back

Last weekend, I worked on my first Github projects: HyperCore, HyperJS, and HyperActive (I like "Hyper" as my project prefix since my name is pronounced "High-pull"). You can find them at: http://github.com/tchype.

In part 1, I'll be covering my original free time prototyping that, a week later, has resulted in the start of something I find very interesting and fun, even if it ends up being pointless...JavaScript-style coding in C#. That first step is something I call HyperDictionary

HyperJS Part 1: HyperDictionary

The original goal of my free-time project was to create a Dictionary of key/value pairs that supports inheritance and overriding of values--a sort of HyperDictionary since it exists in multiple dimensions once you include the ancestors. This had come out of some challenges my friends were facing at work with dealing with customer contracts that had sub-contracts that mostly followed the main contract but had some changes per sub-contract. I wasn't working directly on it myself but I had to scratch that itch at home...

I created a HyperDictionary class that implements IDictionary<string, object>. It allows you to specify a "parent" IDictionary<string, object>. It also allows you to Add, Remove, or Extend values based on their keys and types. You can Add or Remove any key from the dictionary and you can Extend any IEnumerable.

I was inspired by two things:

1. ASP.NET Web.config: The AppSettings section lets you add and remove key/value pairs at the machine level, your web site level, or a particular folder within a web site, always referring back to the higher level to find settings.
2. JavaScript: Prototype inheritance seemed to be the way to go, where you can see if a dictionary "HasOwnProperty(name)" and if not, asking for a key will then check it's parent to see if it has it, and so on.

Examples

In this first example, we simply define a top-level dictionary--top--and a second level dictionary--second--where one property is only defined in top and the other is overridden in second:

Console.WriteLine("Using HyperDictionary to show dictionary inheritance\n==================================================");
var top = new HyperDictionary("top");
top["eyes"] = "brown";
top["hair"] = "pointy";

var second = new HyperDictionary("second");
second.InheritsFrom(top);
second["hair"] = "straight";

Console.WriteLine("top[\"eyes\"]:\t{0}", top["eyes"]);
Console.WriteLine("top[\"hair\"]:\t{0}", top["hair"]);
Console.WriteLine("second[\"eyes\"]:\t{0}", second["eyes"]);
Console.WriteLine("second[\"hair\"]:\t{0}", second["hair"]);

Which outputs:

Using HyperDictionary to show dictionary inheritance
==================================================
top["eyes"]:    brown
top["hair"]:    pointy
second["eyes"]: brown
second["hair"]: straight

This works n-levels deep as well. Here's an example where I removed a setting and extended an IEnumerable as well:

//Extends and removes using an IEnumerable<object> value
top["things"] = new string[] { "first thing", "second thing" };

var third = new HyperDictionary("third");
third.InheritsFrom(second);
third.RemoveProperty("hair");
third.ExtendProperty("things", new object[] { 3, 4, 5 });

//Output members of third - note the absence of "hair" member
Console.Write("third members:\n");
foreach (object o in third)
{
    Console.WriteLine(o);
}
Console.WriteLine();

// Output the extended list of items in "things", 
// some from top and some from third.
// And notice: DIFFERENT DATA TYPES!
Console.Write("third things:\t");
var things = third["things"] as IEnumerable<object>;
foreach (object thing in things)
{
    Console.Write(" | " + thing.ToString());
}
Console.Write(" | ");
Console.WriteLine();

Which outputs:

third members:
[things, System.Linq.Enumerable+<UnionIterator>d__88`1[System.Object]]
[eyes, brown]

third things:    | first thing | second thing | 3 | 4 | 5 |

Potential Uses

Here's a couple quick ideas for uses of the HyperDictionary:

• Versioned object serialization: Each version of the object is it's own HyperDictionary that "inherits" from the previous version and it's own KeyValuePairs the deltas. You can store the Tuples as the values (PropertyActions are Add, Remove, Extend) for each version and easily recreate point-in-time versions. You could also store the "current" full dictionary (including ancestor key/value pairs) separately for speed since going back versions is usually a special-case. This works fairly easily with both a no-SQL and a SQL datastore.
• In-memory data changes: You can read key/value pairs out of a data store (database, cookie, etc.) into a "current settings" dictionary. Then, create a "user changes" dictionary with the "current settings" as the parent. When the user makes changes, it updates the "current settings" dictionary and then you can save the "user settings" dictionary back to the data store, using the proper Add, Remove, or Extend action. Create the "Default Settings" as the top-level dictionary and you can enable "Restore Defaults" very easily.
• Hierarchical configuration: It was partially based on this concept, should work...
• Role-based security: You could implement a base set of entitlements that are set to some default key/value pairs (e.g., "CanUseSystem" => true, "CanAuthorizePayments" => false, "CanCreateUsers" => false) and then, based on the user's role or particular account, add or remove or extend their capabilities using a new dictionary at each level.

Next Up: JavaScript Style

In the next post, I'll describe how I wanted to be able to set items using the indexer notation and access that value via a dynamic property reference:

foo["bar"] = "baz";
var gotBar = foo.bar;

From there, it ends up being an easy cognitive jump to JavaScript style (and, you get to use closures in cool ways)! It also ends up to not be an easy jump to implement it "for reals."

HyperDictionary is part of my HyperCore GitHub project that also includes the capabilities I'll describe in the next two posts. http://github.com/tchype

Ma! I'm On the (Internet) Radio!

I spend dozens of hours a week listening to podcasts (one of the reasons I love Google Listen on my Nexus One). One of the pioneers of podcasting is Pwop Studios and their long-running .NET Rocks! podcast.

This has been one of my favorites for many years and finally, after all of this time, I get a few minutes of fame. You can listen for yourself here:
  .NET Rocks! show 565 - Steve Evans Compares Amazon and Azure.

The podcast gets about 2 million downloads a month, so I am very excited to contribute to such a popular show, even if for only 3 minutes or so. My email starts about 7 minutes in, but if you are a .NET developer, please listen to the whole show. It is a free resource that has contributed greatly to my professional growth over the last several years and I'm very grateful that they provide this kind of content for free. On top of that, there's great humor and real people on this show that really keep you coming back. Shows like this make my long commute bearable and educational!

I'm particularly interested in finishing listening to this episode as I am very interested in the different emerging cloud platforms, and these guys are terrific at drawing out the content that gives you enough of a base understanding so that you know enough to learn more in a targeted, informed fashion.

http://www.dotnetrocks.com/default.aspx?showNum=565

And now, my 15 minutes are up.

Cross-Domain AJAX and JavaScript Techniques

How To Impress Your Friends, Make Money, and Be Cool Like Google (or a Sneaky Spy Like Facebook)

Are you looking to expose your services to the widest audience possible? Are you trying to find a clear description of when to use any of the following techniques you found when you Googled for Cross-Domain JavaScript or AJAX?

• Beacons or Tracking Pixels
• Server-Side Ajax Proxies
• JSONP (JSON with Padding) or Dynamic Script Injection
• iframes to Display Data or Panels
• Using iframes for Cross-Domain Communication
• Domain-Lowering with iframes
• window.postMessage()
• Embedding Flickr, Twitter, RSS feeds, or Google Calendar with only simple JavaScript snippets

Well, this is for you! I recently gave a presentation at work to nearly all of our engineering staff, some of our QA and front-line support folks, and to anyone else who accidentally ended up in our training room during lunch time. It has received great interest since the talk, so I thought I'd unleash it into the wild and see if it helps anyone else! I have shared the presentation on Google Docs at: Cross-Domain AJAX and JavaScript Techniques.

Be sure to review the speaker's notes, as I have a lot of references in there. A few of them refer to an ASP.NET MVC project I used to demo some of the techniques, but I have not included that demo site on this site yet. Most of the interesting ones link to sites that display the techniques in use or have good tutorials on them.

A Cookbook of What and When

There are plenty of sites that explain how to implement many of the cross-domain communication techniques that are available, but I never really saw any that talk about when it is appropriate to use which techniques. That, to me, is the key: understanding my need and circumstances and finding one or more appropriate ways to solve it. The presentation includes a Cross-Domain AJAX and JavaScript Cheat Sheet at the very end that lists the what and when in a condensed format.

Because many of the engineers I work with are more comfortable in the middle-tier services than with front-end development, I include a brief introduction/history of some of the key pieces and the motivation for learning them now. I also cover other lead-in topics such as:

• Limited Scalability of Server-Side Rendering
• "Free" Horizontal Scaling with JavaScript
• AtomPub, gData, OData, E-Tags
• Caching for pages as well as data with various intermediaries
• Facebook Fan panels, Like Buttons, AdSense and other "embeddable" technologies

What's the Business Value of Cross-Domain JavaScript?

As I have been working on more complex sites and pushing for JavaScript-based logging at our company to reduce the technical overhead and reliance on particular server-side technologies for logging, I have had to examine what other companies are doing to scale their services separate from their web site, enable cross-domain sharing of user information and, and how to syndicate and expose the services we provide to other web sites with minimal implementation friction.

It turns out that once companies like Google and Facebook have a valuable repository of information and provide value-add to them, a NEW opportunity arises: SYNDICATION. It is hard to drive traffic to your site beyond some typical growth patterns, advertising, etc. But if you can get others to embed you on their sites, they do the distribution for you and your data, services, and user experience can travel all over the web to where people already are.

The AdSense ads on this very blog are a perfect example of that (they are also about the only secure ad display platform out there). The (now somewhat loathed) Facebook Connect, Facebook Like buttons and iframes that let Facebook see you as you travel across the web are all heavily used, although many engineers are surprised when they find out exactly how they are implemented with "crude, old-school crap" like iframes and domain tricks. I personally find these techniques elegant uses of the standards as documented and HTML5 is expanding on these commonly used techniques to allow more of this type of communication!

Please Enjoy, Comment, and Share the Link

Please feel free to share the link around and to comment on this blog with any and all feedback. This is the kind of thing I would like to have more complete example projects around if it is helpful, or links to other good posts with implementation details. I am really listening to the commentary, so please, feed me :-)

I really feel strongly that these types of techniques drive some of the most successful companies into incredible growth, visibility, and utility and we can all benefit from learning these techniques, when to use them, and why they are awesome.

Being a Developer is Fun Again!

Or, "What I Did on My Summer (Fall, Winter) Vacation"

It has been seven months since my last confession. No, I didn't fall into the "no one reads my blog so screw it" mode that so many fall into after starting the next great web log. Instead, I was busy rediscovering how much fun it is to be a developer and working Insane Hours™ to create cool things!

On top of that, I discovered more of what it was like to be in a community to work with great technology to create said cool things, so the Insane Hours™ did have a lot of joy in them...

Quick Exposition

There was a time in my life, about 6 years ago, where I thought that being a developer was a bad idea. I figured all of the really interesting problems had been solved and software engineering was going to be about either assembling systems from components that were already built, or building those components in the latest fad language--the only difference being syntax. I was also told I could be outworked when 3 offshore people in a different economy were put together they were still cheaper than me and could do what I could do. I even went so far as to go into development management, project management, and eventually business analysis and stopped coding full-time for a couple years.

After realizing that I hated doing everything around development and not actually participating in the development myself--and that outsourcing was not delivering what was promised, I took a substantial pay cut and went back to coding 90% or more of my time. The first two years were fun: becoming proficient in C# and .NET (participating in the VS 2005 and SQL 2005 Beta programs), and then working on WAP and XHTML Mobile applications, including a critically acclaimed (and Apple Staff Pick) early iPhone web application. Even with all of that, it still wasn't spectacular...

Welcome to Web Development, Post Dot-Com Bust

I have been able to do more and more web development, including middle-tier services but focusing on gaining those front-end skills that I once loathed, since any lame-o that could write basic HTML tags using Microsoft Front Page could get a high-paying job back in the early days.

What I Enjoy Most

All of this crazy work the last several months has exposed me to (more than) full-time exposure to a space that is very exciting to me, and should be exciting to developers looking for something cool to work on! Here are the highlights:

• Dynamic and Open Source Languages
  • JavaScript
  • Python
  • Ruby
  • Relearning VIM, Emacs (and why I didn't use them much in the past anyway)
• Separation of Concerns in Web Development (a.k.a. The Death of ASP.NET Web Forms)
  • CSS/DIV-Based Layout
  • MVVM, MVP, MVC
  • Silverlight
  • Ruby on Rails
  • ASP.NET MVC
• The Browser as Runtime Virtual Machine
• JavaScript, jQuery, iFrames, and window.postMessage
• Google Web Toolkit
• Community
  • Stack Overfow
  • Scott Hanselman
  • Phil Haack
  • GOOD Podcasts: .NET Rocks, Hanselminutes, DOCTYPE, TWiT network with TWiT and TWiG and Leo Leporte (screw you, Mike Arrington!)
  • Yes, Twitter and Facebook
  • Lot's and lot's (and lot's) of phone screens, in-person interviews, and hiring of interns and junior developers!
  • Great Co-Workers/Partners
  • Terrific Wife and Kids

Overall, these things have contributed to a HUGE cognitive leap for me, probably the biggest one-year growth that I've had as a professional. Hiring interns has been very rewarding and is a good reminder that I want to hire people that are better than I was coming out of school!

I will likely expand on a few of these points in posts in the near future. In the meantime, you can check out the result of what I worked on during the summer and fall and winter of 2009-2010 with a dozen other folks at http://www.haggle.com!