Archive for the SqueakJS Category

Backend Caffeine with Node.js and Tweetcoding

Posted in Appsterdam, consulting, Context, Smalltalk, Spoon, SqueakJS with tags , , , , , , on 27 June 2017 by Craig Latta

tweetcodingWe’ve seen a couple of examples of frontend Caffeine development, using jQuery UI and voxel.js. Let’s look at something on the backend, using Node.js and something I like to call tweetcoding.

Social Livecoding

Livecoding means having all the capabilities of your development environment while your system is live. Runtime is all the time. Smalltalk programmers have always enjoyed this way of working. It is particularly effective in domains such as music performance, where the ability to make changes while your system is running is critical, both for fixing bugs and for artistic expression. One of my artistic goals with musical livecoding (writing code that makes live music for an audience) is audience accessibility. I want the audience to have some understanding of what’s going on in my code, even if they aren’t coders (yet).

Another aspect I like to incorporate is audience participation. If the audience actually does understand what I’m doing in my code, why not let them join in? Social media has become a ubiquitous communication platform; billions of people are using it with prose and pictures. I think the time has come to use it with live code as well. Nearly everyone in the audience has a powerful computer with social media access in a pocket. We can all livecode together.

Let’s Do This

We need three things to make this happen: a way for people to submit code, something that listens for that code and evaluates it, and a shared artifact everyone can see that shows the code’s effects. I like Twitter as a coding medium, for its minimalism. It also has a culture of hashtag use, and a streaming API, which make listening for code straightforward. For the shared artifact, we can use something viewed with Caffeine in a web browser.

For our first example, let’s use something simple for our coding domain, like turtle graphics. A person can tweet instructions to a turtle which draws on a square piece of paper, 100 units on a side. There are four commands:

  • color, for the turtle to change its pen for one of the given color
  • down, for the turtle to put its pen down on the paper
  • rotate, for the turtle to change direction
  • move, for the turtle to move some distance
  • up, for the turtle to take its pen up from the paper

We’ll say that that the turtle can be identified by some ID, and is initially at the center of the paper. A tweetcoded instruction for the turtle might look like:

FranzScreenSnapz002

Tweetcoding tweets start with the #tweetcoding hashtag, so people seeing one for the first time can look at others with that hashtag, to see what it’s about. This is followed by hashtags identifying our turtle-graphics protocol, and the turtle we’re using. We end the tweet with a drawing instruction, using our instruction set above.

Livecoding NodeJS

To detect these tweets, we’ll use the Twitter streaming API from Node.js. In the spirit of livecoding, we won’t use an application-specific Node.js module for this directly, but instead inject JavaScript code live into a Node.js instance, from Caffeine in a web browser. I’ve written a node-livecode Node.js module which takes commands over a websocket. It starts with three instructions:

  • require, for loading another Node.js module into itself with npm
  • add instruction, for adding an instruction to itself
  • eval, for evaluating JavaScript code

You can see the implementation at GitHub. It can use SSL to keep injected code secure in transit, so you may want to set up a certificate for your server with Let’s Encrypt. Also note that it exposes the listening server as a global variable, so that you can use it in your injected code.

Once we have node-livecode on a server listening for commands, we can inject code into it from Caffeine. First we’ll inject code to listen to #tweetcoding tweets from Twitter:

| websocket |

websocket := JS WebSocket newWithParameters: {'wss://yourserver:port'}.
JS top at: #websocket put:websocket.

websocket
  at: #onmessage
  put: [:message |
    Transcript
      cr;
      nextPutAll: message data asString;
      endEntry];
  at: #onopen
  put: (
    (JS Function) new: '
      window.top.ws.send(JSON.stringify({
        credential: ''shared secret'',
        verb: ''require'',
        parameters: {
          package: ''node-tweet-stream'',
          then: ''
            var Twitter = require(''node-tweet-stream'')
              , twitter = new Twitter({
                  consumer_key: '''',
                  consumer_secret: '''',
                  token: '''',
                  token_secret: ''''})

            twitter.on(
              ''tweet'',
              function (tweet) {
                if (instructions[''broadcast'']) {
                  instructions[''broadcast''](tweet)}}

            twitter.on(
              ''error'',
              function (err) {
                console.log(''error in uploaded code'')})

            twitter.track(''#tweetcoding #turtlegraphics '')''}})))

The example above uses handwritten JavaScript code, but we could also use Caffeine’s JavaScript code generation to produce it from Smalltalk code (see the method MethodNode>>javaScript for the implementation). Also, since we’ll be injecting several things, it would be nice to have a more compact way of writing it. Let’s move that workspace code to a class.

Next we’ll inject a new instruction, for initializing a set of drawing-command listeners:

| client |

client := (
  NodeJSLivecodingClient
    at: 'wss://yourserver:port'
    withCredential: 'shared secret').

client
  addInstruction: 'initialize listeners'
  from: '
    function () {
      global.listeners = []
      return ''initialized listeners''}'

To round out our tweetcoding protocol, we’ll add instructions for accepting listeners, and broadcasting tweets detected from Twitter:

client
  addInstruction: 'start listening'
  from: '
    function () {
      listeners.push(this)
      return ''added listener''}'
  addInstruction: 'stop listening'
  from: '
    function () {
      listeners.splice(listeners.indexOf(this), 1);
      return ''removed listener''}'
  addInstruction: 'broadcast'
  from: '
    function (payload) {
      listeners.forEach(
        function (listener) {listener.send(payload)})
      return ''broadcasted''}'

Putting It All Together

Now we can write other Smalltalk classes which subscribe to tracked tweets from the server, and collaborate to do something interesting with them in the web browser. I’ve implemented the following classes:

Object
  NodeJSLivecodingClient
    TweetcodingClient
      TurtleGraphicsTweetcodingClient
  Turtle
  Tweet

Instances of NodeJSLivecodingClient can inject code into a node-livecoding server, and invoke code added by other clients. Instances of TweetcodingClient can also set tracking filters for tweets, and process matching tweets when they occur. Instances of TurtleGraphicsTweetcodingClient can also control Turtles, which can draw on canvases in Caffeine windows. Instances of Tweet bundle up the text and metadata of tweets. For the implementation, clear your browser’s cache for caffeine.js.org (including IndexedDB), and reload https://caffeine.js.org/.

I’m also running a node-livecoding server injected with turtle-graphics tweetcoding code, at wss://frankfurt.demo.blackpagedigital.com:8087/. Once you get connected (and tweets are flowing), you might see something like this:

Google ChromeScreenSnapz014

After developing this system, I’ve realized I don’t really need to run SqueakJS from within NodeJS; just injecting code into it is fine. There are many possibilities here. :)

Have fun, and please let me know what you think!

 

Advertisements

Caffeine in 3D with voxel.js

Posted in Appsterdam, consulting, Context, Naiad, Smalltalk, Spoon, SqueakJS with tags , , , , , , , on 24 June 2017 by Craig Latta

voxel.js in CaffeineSince Caffeine is powered by SqueakJS, you can create mashups with any other JavaScript frameworks you like. Let’s take a simple look at 3D graphics using voxel.js, an open-source voxel game building toolkit.

hello blocks

Following the “hello world” example at voxeljs.com, we generate a simple Minecraft-like blocks world in which we can walk around and dig (you can visit it here). The example gives us a JavaScript file, builtgame.js, that we can also use from Caffeine.

As generated, builtgame.js evaluates its createGame function at load time. This creates an HTML5 canvas, initializes WebGL, and begins the game when the hosting page is loaded. We want to save those steps for SqueakJS to initiate, and we also want to use a canvas of our own, in a Caffeine window.

hooks for Caffeine

We can achieve the first part by changing builtgame.js so that it just puts createGame somewhere SqueakJS can get to it, instead of evaluating it. We can create a property on the browser DOM window for this:

window.game = createGame;

Normally we would edit the source projects from which builtgame.js is generated, rather than builtgame.js directly (properly forking the corresponding repositories), but for this example we’ll just go ahead.

Voxel.js uses the three.js framework as its WebGL interface. The three.js WebGL renderer accepts an HTML5 canvas parameter for its initialization function. The second change we’ll make to builtgame.js is to pass a canvas set by SqueakJS in another window property:

View.prototype.createRenderer = function() {
  this.renderer = new THREE.WebGLRenderer({
    canvas: window.gameCanvas !== undefined ? window.gameCanvas : undefined,
    antialias: true});
  this.renderer.setSize(this.width, this.height);
  this.renderer.setClearColorHex(this.skyColor, 1.0);
  this.renderer.clear();};

Finally, we’ll change the game rendering initialization function, to save a reference to the voxel.js renderer’s event emitter, so that we can tell it to pause from SqueakJS:

Game.prototype.initializeRendering = function(opts) {
  var self = this;
  if (!opts.statsDisabled) self.addStats();
  window.addEventListener('resize', self.onWindowResize.bind(self), false);
  self.ee = (
    requestAnimationFrame(window).on(
      'data',
      function(dt) {
        self.emit('prerender', dt);
        self.render(dt);
        self.emit('postrender', dt);
      });
  if (typeof stats !== 'undefined') {
    self.on(
      'postrender',
      function() {
        stats.update();});}}

on the Smalltalk side

Now, in SqueakJS, we can create a VoxelJS class:

Object
  subclass: #VoxelJS
  instanceVariableNames: ''
  classVariableNames: ''
  poolDictionaries: ''
  category: 'Hex-HTML5-WebGL-VoxelJS'.

VoxelJS class instanceVariableNames: 'game'

We’ll give it a class-side method to load and initialize voxel.js, with a Caffeine window and canvas for it to use:

initialize

| canvas |

Webpage current loadScriptFrom: 'js/voxeljs/builtgame.js'.
canvas := Webpage createWorldOfKind: 'voxeljs'.
canvas styleAt: #borderRadius put: '10px'.

(Webpage current)
  windowizeElementNamed: canvas window id
  closingWith: [
    self pause.
    Webpage current top at: #gameCanvas put: nil].

canvas window dragWith: canvas window windowButtonsTray moveButton.
game := (
  (Webpage current top)
    at: #gameCanvas put: canvas;
    game: {#container -> canvas window})

We’ll also add a method for pausing the voxel.js renderer, using the ee property we added to the game rendering initialization in builtgame.js:

pause
  game ee pause.
  (JQuery at: #fps) element remove

In a workspace, we send our initialization message:

VoxelJS initialize

Now we have our first voxel world, running in a Caffeine window that we can easily close, rather than the whole screen. If you clear your browser cache (including IndexedDB) for caffeine.js.org, you can reload the Caffeine page to see this code in action.

Please let me know if you get this far!

 

Caffeine :: Livecode the Web!

Posted in Appsterdam, consulting, Context, Naiad, Smalltalk, Spoon, SqueakJS with tags , , , , , , , , , , , , , , on 22 June 2017 by Craig Latta

CaffeineFor the impatient… here it is.

Back to the Future, Again

With the arrival of Bert Freudenberg’s SqueakJS, it was finally time for me to revisit the weird and wonderful world of JavaScript and web development. My previous experiences with it, in my consulting work, were marked by awkward development tools, chaotic frameworks, and scattered documentation. Since I ultimately rely on debuggers to make sense of things, my first question when evaluating a development environment is “What is debugging like?”

Since I’m a livecoder, I want my debugger to run in the web browser I’m using to view the site I’m debugging. The best in-browser debugger I’ve found, Chrome DevTools (CDT), is decent if you’re used to a command-line interface, but lacking as a GUI. With Smalltalk, I can open new windows to inspect objects, and keep them around as those objects evolve. CDT has an object explorer integrated into its read-eval-print loop (REPL), and a separate tab for inspecting DOM trees, but using them extensively means a lot of scrolling in the REPL (since asynchronous console messages show up there as well) and switching between tabs. CDT can fit compactly onto the screen with the subject website, but doesn’t make good use of real estate when it has more. This interrupts the flow of debugging and slows down development.

The Pieces Are All Here

With SqueakJS, and its JavaScript bridge, we can make something better. We can make an in-browser development environment that compares favorably with external environments like WebStorm. I started from a page like try.squeak.org. The first thing we need is a way to move the main SqueakJS HTML5 canvas around the page. I found jQuery UI to be good for this, with its “draggable” effect. While we’re at it, we can also put each of Squeak‘s Morphic windows onto a separate draggable canvas. This moves a lot of the computation burden from SqueakJS to the web browser, since SqueakJS no longer has to do window management. This is a big deal, since Morphic window management is the main thing making modern Squeak UIs feel slow in SqueakJS today.

SqueakJS provides a basic proxy class for JavaScript objects, called JSObjectProxy. Caffeine has an additional proxy class called JSObject, which provides additional reflection features, like enumerating the subject JS object’s properties. It’s also a good place for documenting the behavior of the JS objects you’re using. Rather than always hunting down the docs for HTMLCanvasElement.getContext on MDN, you can summarize things in a normal method comment, in your HTMLCanvasElement class in Smalltalk.

Multiple Worlds

With a basic window system based on HTML5 canvases, we can draw whatever we like on those canvases, using the SqueakJS bridge and whatever other JS frameworks we care to load. I’ve started integrating a few frameworks, including React (for single-page-app development), three.js (for WebGL 3D graphics development), and morphic.js (a standalone implementation of Morphic which is faster than what’s currently in Squeak). I’ll write about using them from Caffeine in future blog posts.

Another framework I’ve integrated into Caffeine is Snowglobe (for Smalltalk app streaming and other remote GUI access), which I wrote about here previously. I think the Snowglobe demo is a lot more compelling when run from Caffeine, since it can co-exist with other web apps in the same page. You can also run multiple Snowglobes easily, and drag things between them. I’ll write more about that, too.

Fitting Into the JavaScript Ecosystem

To get the full-featured debugger UI I wanted, I wrote a Chrome extension called Caffeine Helper, currently available on the Chrome Web Store. It exposes the Chrome Debugging Protocol (CDP) support in the web browser to SqueakJS, letting it do whatever the CDT can do (CDT, like SqueakJS, is just another JavaScript-powered web app). The support for CDP that I wrote about previously uses a WebSocket-based CDP API that requires Chrome to be started in a special way. The Caffeine Helper extension provides a JavaScript API, without that requirement.

I also wrote support for generating Smalltalk code from JavaScript, using the esprima parsing framework, and vice-versa. With my debugger and code generation, I’m going to try developing for some file-based JS projects, using Smalltalk behind the scenes and converting to and from JavaScript when necessary. I think JS web development might actually not drive me crazy this way. :)

Please Try It Out!

So, please check out Caffeine, at caffeine.js.org! I would very much appreciate your feedback. I’m particularly interested to hear your use cases, as I plan the next development steps. I would love to have collaborators, too. Let’s build!

%d bloggers like this: