Messing with CPS in JavaScript

This might sound like bullshit but bear with me please.

I wont explain what is Continuation Passing Style.

The example

Imagine that we need to create a function to retrieve the arrival date for a given flight number and departure date. The departure date is optional and if not specified, then we have to fetch the last flight with the flight number first.

Something like this:


/*
 * Fetch the arrival date for a given flight and departure date.
 * If departure date is not specified, then return last flight arrival date.
 */
function getArrivalDate(flightNumber, departureDate){
  if(!departureDate){
    departureDate = request.getJson("http//airline.com/api/flights/" + flightNumber + "/last/").departureDate;
  }
  return request.getJson("http//airline.com/api/flights/" + flightNumber + "/schedules/" + departureDate).arrival;
}

The problem is that this code is synchronous and is not the way you usually work in javascript. So, you might change to something like this:


function getArrivalDate(flightNumber, departureDate, callback){
  if(!departureDate){
    request.getJson("http//airline.com/api/flights/" + flightNumber + "/last/", function(err, flight){
      request.getJson("http//airline.com/api/flights/" + flightNumber + "/schedules/" + flight.departureDate, function(err, flightSchedule){
        callback(flightSchedule.arrival);
      });
    });
  }else{
    request.getJson("http//airline.com/api/flights/" + flightNumber + "/schedules/" + departureDate, function(err, flightSchedule){
      callback(flightSchedule.arrival);
    });
  }
}

There is duplication in the above code, let's try to apply DRY:


function getArrivalDate(flightNumber, departureDate, callback){

  //But departureDate here is not optional, did you hear me?
  function calc(flightNumber, depDate, callback){
    request.getJson("http//airline.com/api/flights/" + flightNumber + "/schedules/" + depDate, function(err, flightSchedule){
      callback(flightSchedule.arrival);
    });
  }

  if(!departureDate){
   request.getJson("http//airline.com/api/flights/" + flightNumber + "/last/", function(err, flight){
      calc(flightNumber, flight.departureDate, callback);
    });
  }else{
    calc(flightNumber, departureDate, callback);
  }
}

Now is better. By sure you can do something better than this using one of the gazillions of control-flow libraries for javascript and for node or using something like promises.

But for the sake of this experiment let's try to do something different now.

A CPS IF statement

With a function like this:


var flow = {
  if: function(condition, ifTrue, next){
    if(condition){
      ifTrue(next);
    }else{
      next();
    }
  }
};

then we can define our getArrivalDate as this:


function getArrivalDate(flightNumber, departureDate, callback){
  flow.if(!departureDate, function(next){
    request.getJson("http//airline.com/api/flights/" + flightNumber + "/last/", next);
  }, function(err, flight){
    var depDate = departureDate || flight.departureDate;
    request.getJson("http//airline.com/api/flights/" + flightNumber + "/schedules/" + depDate, function(err, sched){
      callback(sched.arrival);
    });
  });
};

The first callback: fetch the last flight is executed only if the condition departure date is nully is true. The second callback is executed right away if the condition is false or when the first callback fire it if the condition is true.

This simple example reads better than the standard IF and in my experience it makes much more sense with more complicated things.

What we did?

What I did was to create an alternative IF construct with Continuation Passing Style and then what I accomplish is to use everything in this function as Continuation Passing Style. You can do the same for loops for instance (serial / parallel).

In other words I just bring LISP to js.

The other way!

Another way is to avoid CPS at all. If you look at:

All those languages has an alternative syntax to execute asynchronous methods without continuation passing style and GRANTED you will find that those play nice with the non-cps constructs.

Conclusion

I think work flows with CPS invocation are better and easy to follow with continuation passing style statements than with the standard constructs.

In my humble opinion this is one common reason some people have a hard time to switch from synchronous to javascript- asynchronous. Which should be read as Direct Style vs Continuation Passing Style instead. Often people thinks that the problem is callback nesting but that one is easy to solve.

Another interesting idea will be to create a language with an alternative syntax to execute async methods (defined as CPS) close to the "direct style" that compiles down to JavaScript LISPished. I think this is exactly what Iced CoffeeScript and StratifiedJs already do.

Posted in javascript node | 0 Comments and 0 Reactions

node.js and the beauty of working on a platform that embraces opensource

In this article I will show one of the most beautiful things of working with node.js and how noders do open source in the right way.

I have been working full time in a project using node.js for Tellago Studios, I have read a lot of posts about node.js being cancer, why node.js sucks, why ruby is better, why javascripts callbacks for async workflows are complex and so on, and even if I agree with some things at some point there are things in node.js done right, things to applaud and things to learn from it. This article is about something to applaud.

tl;dr

When working with node.js you can swap a dependency or a sub-dependency with a link to your fork-branch in a git repository. Make sure you understand the [1] algorithm for loading modules and [2] npm install

A bug story

Lets say you are working on your project and you are using a third party opensource module named Foo, your package.json will looks like:


{
  "name": "myproject",
  "version": "0.0.1",
  "dependencies": {
    "foo" : "*"
  },
}

The star * here means any version.

If you look at the your project folder you will see a tree like this:


myproject/
  server.js
  package.json
  node_modules/
    foo/
      package.json
      node_modules/
        bar/
          package.json
          node_modules/
            another/

Now suppose that you find a bug, you fire up [3] node-inspector and you start digging in your modules, then you figure out that the problem is in the module "bar".

Now, the flow will be something like this:

  1. Clone the module Bar (which of course is opensource of course as the bast majority of node.js modules) in another folder. If you don't know where the Bar module repository is hosted, don't worry, look at its package.json the url will be there for sure.
  2. Fix the issue and try things as crazy. Write tests!
  3. Commit, revert, commit, branch blah
  4. Push to your fork of the project
  5. Do a pull request to the project owner[s]

So far these are the standards steps you will take in any platform, even in .net. Because node.js developers take so serious this opensource business it is very likely that your pull request will be at least reviewed or discussed the same day.

But what if not?

You still have a bug to solve in your project, and no one in your team or company want to wait for you or other people to:

  1. merge your fix
  2. publish to npm a fix of Bar
  3. publish a new version of Foo pointing to the fixed Bar.

Node community understand open source

As crazy as it might sound, we can easily do this in OUR package.json:


{
  "name": "myproject",
  "version": "0.0.1",
  "dependencies": {
    "bar": "git://github.com/youruser/bar.git#my-branch-to-the-ugly-bug"
    "foo" : "*"
  },
}

run npm install and you are done. Your project and most important, your local instance of the package Foo is using the your fork of Bar. Your team will be able to work with and everyone will be happy. We can even omit the pull request if we find that is something useful only in our project or something that should not be on the main repository.

We can also release our project as opensource and no one is going to be affected.

Whats the price for a feature like this in a programming language / platform?

How does it works?

There are few things that make this possible.

The first thing is the ability of npm to install things from a git repository as explained in [2] npm install the package could be:

a) a folder containing a program described by a package.json file
b) a gzipped tarball containing (a)
c) a url that resolves to (b)
d) a <name>@<version> that is published on the registry with (c)
e) a <name>@<tag> that points to (d)
f) a <name> that has a "latest" tag satisfying (e)
g) a <git remote url> that resolves to (b)

The second things is how the module system (i.e. the require function) works as explained in [1] Node.js: Modules - All together , specifically this part:

NODE_MODULES_PATHS(START)
1. let PARTS = path split(START)
2. let ROOT = index of first instance of "node_modules" in PARTS, or 0
3. let I = count of PARTS - 1
4. let DIRS = []
5. while I > ROOT,
   a. if PARTS[I] = "node_modules" CONTINUE
   c. DIR = path join(PARTS[0 .. I] + "node_modules")
   b. DIRS = DIRS + DIR
   c. let I = I - 1
6. return DIRS

It means that when trying to solve the module Bar from Foo, this thing will start with a path like this:

node_modules/foo/node_modules

and as is explained there, first it will look for Bar in the first node_modules folder.

And last but not least is the fact that the node.js community understand how opensource works. All modules in node.js are opensource, and almost all modules in node are in git and in github. It is easy to understand modules because all modules have almost the same file structure package.json, lib/bin folders, etc. This means there is a way to do it and most people is following that way.

I have the pleasure to assist to jsConf in Argentina and listen to [4] Isaac Z. Schlueter current curator of node.js and creator of npm and it is not surprising that the technical thing explained here is in the same line that the principles he encourage. Unfortunately the talk is not online (yet? saw it was being recorded) neither his slides but I found slides from another very similar talk [5] TacoConf Anarchism

Posted in node javascript | 0 Comments and 0 Reactions

Writing your first native module for node.js on Windows

In this article I will guide you on how to write a native module for node.js in Windows using Visual Studio.

About the example

In this example we are going to build a module to log events to the Windows Event Log. The project is already published in [1] github and you can install it with npm.

I like it, but why native?

There are several reasons why you would build a native extension for node.js, but for me the most important reason is when you want to use an API that doesn't have an open or well documented protocol and the vendor only provides a native library. Of course it can also happen that the protocol is well documented and open but it was easier to interface the native implementation than writing a javascript client from scratch (E.g. the mongodb package). In this case we are talking about the Windows Event Log which is a component of the operative system.

I don't speak c++

The code that I am going to show well... it might have several problems. The thing is that my c++ skills are close to my English skills. I can read it and I can write it but as Russ Olsen say (for ruby) I have not absorb the "c++ way" of thinking and problem solving. So, please if you think that something could be improved just tell me.

C++/CLI

C++/CLI is a Microsoft's evolution of the c++ language invented by Bjarne Stroustrup. The name is a tuple "C++" and "CLI", CLI means "Command Language Infrastructure" which is the specification for the .Net framework. So, you can think of c++/cli as:

  • The way you write c++ for the .Net framework
  • Interoperability between the c++ model and the .net framework In this examples I will use the .Net class [2]System.Diagnostic.EventLog.

Configuring the environment

The first thing you will need to do is to download the [3]Node.js source code and uncompress it somewhere in your disk. Then you will have to build the source code by running the vcbuild.bat file. Note: the build process needs python 2.x installed and it should be accesible from the path.

Creating the project in Visual Studio

1-Create a new "CLR Empty Project":

2012-04-10_0953.png

2-Do these changes to the project settings:

2012-04-10_0956.png

3-Add these directories to the Include Directories (remember to change C:\node-v0.6.14\ with the path where you have the node.js sources)

2012-04-10_1003.png

4-Change the Libraries Directories as follows:

2012-04-10_1008.png

5-Add a new "C++ File (.cpp)" called "EventLog", so your solution has to look as:

2012-04-10_1043.png

Note: that the cpp file that we add has to be /CLR.

Show me the code

Nitpicker corner: Don't get overwhelmed! I wrote some comments in the source to explain the different pieces:


#pragma comment(lib, "node")

#using <mscorlib.dll>
#using <system.dll>

#include <node.h>
#include <v8.h>
#include <string>
#include <gcroot.h>
#include <string>
#include <iostream>
#include <uv.h>

using namespace node;
using namespace v8;

class EventLog : ObjectWrap
{
private:
    //the field that will hold an instance of System.Diagnostics.EventLog
    gcroot<System::Diagnostics::EventLog^> _eventLog;

public:
    
    static Persistent<FunctionTemplate> s_ct;
    
    //module initialization
    static void NODE_EXTERN Init(Handle<Object> target)
    {
        HandleScope scope;

        // set the constructor function
        Local<FunctionTemplate> t = FunctionTemplate::New(New);

        // set the node.js/v8 class name
        s_ct = Persistent<FunctionTemplate>::New(t);
        s_ct->InstanceTemplate()->SetInternalFieldCount(1);
        s_ct->SetClassName(String::NewSymbol("EventLog"));

        // registers a class member functions 
        NODE_SET_PROTOTYPE_METHOD(s_ct, "log", Log);
        
        target->Set(String::NewSymbol("EventLog"),
            s_ct->GetFunction());
    }
    
    //Constructor
    //Source: the name of the event log we will use
    //LogName: the logName could be Application, System, or a new one.
    EventLog(System::String^ source, System::String^ logName) 
    {
        if(!System::Diagnostics::EventLog::SourceExists(source)){
            System::Diagnostics::EventLog::CreateEventSource(source, logName);
        }
        
        _eventLog = gcnew System::Diagnostics::EventLog();
        _eventLog->Source = source;
    }

    //finalizer, kill the _eventLog.
    //This will call the IDisposable
    ~EventLog()
    {
        delete _eventLog;
    }

    //Transform a v8 argument to a .Net String
    static inline gcroot<System::String^> ParseArgument(Arguments const&args, int argumentIndex)
    {
        Local<String> message = Local<String>::Cast(args[argumentIndex]);
        gcroot<System::String^> m = gcnew System::String(((std::string)*v8::String::AsciiValue(message)).c_str());
        return m;
    }

    //This is the method that call node will call when we do *new EventLog(...)*
    //Note that it looks as a Javascript func since it returns "this"
    static Handle<Value> New(const Arguments& args)
    {
        HandleScope scope;

        if (!args[0]->IsString()) {
            return ThrowException(Exception::TypeError(
                String::New("First argument must be the name of the event log source")));
        }
        if (!args[1]->IsString()) {
            return ThrowException(Exception::TypeError(
                String::New("Second argument must be the name of the event log: (Application, System)")));
        }

        System::String^ s = ParseArgument(args, 0);
        System::String^ ln = ParseArgument(args, 1);

        EventLog* pm = new EventLog(s, ln);

        pm->Wrap(args.This());
        return args.This();
    }

    //The log method that node will call.
    //It unwraps the c++ object and call WriteEntry in the _eventLog field
    static Handle<Value> Log(const Arguments& args)
    {
        if (!args[0]->IsString()) {
            return ThrowException(Exception::TypeError(
                String::New("First argument must be the message to log.")));
        }
        
        if (!args[1]->IsString()) {
            return ThrowException(Exception::TypeError(
                String::New("Second argument must be the type of the entry Information/Warning/Error.")));
        }

        gcroot<System::String^> m = ParseArgument(args, 0);
        gcroot<System::String^> t = ParseArgument(args, 1);
        gcroot<System::Diagnostics::EventLogEntryType> logt = (System::Diagnostics::EventLogEntryType)System::Enum::Parse(System::Diagnostics::EventLogEntryType::typeid, t);

        //unwrap the instance! that's crazy
        EventLog* xthis = ObjectWrap::Unwrap<EventLog>(args.This());

        xthis->_eventLog->WriteEntry(m, logt, 1000);

        return Undefined();
    }
};

Persistent<FunctionTemplate> EventLog::s_ct;

extern "C" {
    void NODE_EXTERN init (Handle<Object> target)
    {
        EventLog::Init(target);
    }
    NODE_MODULE(sharp, init);
}

Some interesting things about this:

  • Node will call always the static methods of the class, the New and the Log.
  • In the statics methods we have to unwrap/wrap the "this" to get/st the real instance of the class (c++ instance).
  • We store the _eventLog instance in a private non static field.
  • Arguments args; is like the javascript "arguments" kind-of-array that you have in every javascript function
  • convert v8 javascript values to c++ values, then convert it to .net values. For instance the strings

Testing the module

When you generate this project, you will have a "debug" folder at the same level than the solution (note that is not the debug folder of the project folder). There is one important file there, the ".node" file which is a dll than node can talk to. Now, if you can open a command line prompt and try the stuff with the node [4]REPL, but remember that in order to write to the event log you need an elevated prompt, otherwise will happen this:

2012-04-10_1212.png So, you run cmd.exe as an administrator, go to the solution folder \debug and do this:

2012-04-10_1216.png

Then if you open the event log you will see this:

2012-04-10_1219.png

Debugging

You can debug your module from visual studio (that's crazy). First you need to be running Visual Studio as an admin, this is not always, it is because we are running the node's REPL as an admin in order to write to the event log. Once you have done the "require" call in the node REPL, go to the Debug menu of Visual Studio and then "Attach to Process" and look at the node process:

2012-04-10_1224.png

Then you can set breakpoints and this will happen:

2012-04-10_1230.png

TODOs

As you can see here we are executing the eventLog.WriteLog method in the same thread. An standard for this is to open a new thread using "uv". I will explain this in another article later.

Credits

  • I have used this guide [4]example from Saar Yahalom as starting point.

References

Posted in node.js javascript | 0 Comments and 0 Reactions

  • Categories

  • Archives