Archive for browser

Livecoding other tabs with the Chrome Remote Debugging Protocol

Posted in Appsterdam, consulting, Context, Smalltalk, SqueakJS with tags , , , , , , , on 24 July 2017 by Craig Latta

Chrome Debugging Protocol

We’ve seen how to use Caffeine to livecode the webpage in which we’re running. With its support for the Chrome Remote Debugging Protocol (CRDP), we can also use it to livecode every other page loaded in the web browser.

Some Help From the Inside

To make this work, we need to coordinate with the Chrome runtime engine. For CRDP, there are two ways of doing this. One is to communicate using a WebSocket connection; I wrote about this last year. This is useful when the CRDP client and target pages are running in two different web browsers (possibly on two different machines), but with the downside of starting the target web browser in a special way (so that it starts a conventional webserver).

The other way, possible when both the CRDP client and target pages are in the same web browser, is to use a Chrome extension. The extension can communicate with the client page over an internal port object, created by the chrome.runtime API, and expose the CRDP APIs. The web browser need not be started in a special way, it just needs to have the extension installed. I’ve published a Caffeine Helper extension, available on the Chrome Web Store. Once installed, the extension coordinates communication between Caffeine and the CRDP.

Attaching to a Tab

In Caffeine, we create a connection to the extension by creating an instance of CaffeineExtension:

CaffeineExtension new inspect

As far as Chrome is concerned, Caffeine is now a debugger, just like the built-in DevTools. (In fact, the DevTools do what they do by using the very same CRDP APIs; they’re just another JavaScript application, like Caffeine is.) Let’s open a webpage in another tab, for us to manipulate. The Google homepage makes for a familiar example. We can attach to it, from the inspector we just opened, by evaluating:

self attachToTabWithTitle: 'Google'

Changing Feelings

Now let’s change something on the page. We’ll change the text of the “I’m Feeling Lucky” button. We can get a reference to it with:

tabs onlyOne find: 'Feeling'

When we attached to the tab, the tabs instance variable of our CaffeineExtension object got an instance of ChromeTab added to it. ChromeTabs provide a unified message interface to all the CRDP APIs, also known as domains. The DOM domain has a search function, which we can use to find the “I’m Feeling Lucky” button. The CaffeineExtension>>find: method which uses that function answers a collection of search results objects. Each search result object is a proxy for a JavaScript DOM object in the Google page, an instance of the ChromeRemoteObject class.

In the picture above, you can see an inspector on a ChromeRemoteObject corresponding to the “I’m Feeling Lucky” button, an HTMLInputElement DOM object. Like the JSObjectProxies we use to communicate with JavaScript objects in our own page, ChromeRemoteObjects support normal Smalltalk messaging, making the JavaScript DOM objects in our attached page seem like local Smalltalk objects. We only need to know which messages to send. In this case, we send the messages of HTMLInputElement.

As with the JavaScript objects of our own page, instead of having to look up external documentation for messages, we can use subclasses of JSObject to document them. In this case, we can use an instance of the JSObject subclass HTMLInputElement. Its proxy instance variable will be our ChromeRemoteObject instead of a JSObjectProxy.

For the first message to our remote HTMLInputElement, we’ll change the button label text, by changing the element’s value property:

self at: #value put: 'I''m Feeling Happy'

The Potential for Dynamic Web Development

The change we made happens immediately, just as if we had done it from the Chrome DevTools console. We’re taking advantage of JavaScript’s inherent livecoding nature, from an environment which can be much more comfortable and powerful than DevTools. The form of web applications need not be static files, although that’s a convenient intermediate form for webservers to deliver. With generalized messaging connectivity to the DOM of every page in a web browser, and with other web browsers, we have a far more powerful editing medium. Web applications are dynamic media when people are using them, and they can be that way when we develop them, too.

What shall we do next?

 

Advertisements

Caffeine in 3D with voxel.js

Posted in Appsterdam, consulting, Context, Naiad, Smalltalk, Spoon, SqueakJS with tags , , , , , , , on 24 June 2017 by Craig Latta

voxel.js in CaffeineSince Caffeine is powered by SqueakJS, you can create mashups with any other JavaScript frameworks you like. Let’s take a simple look at 3D graphics using voxel.js, an open-source voxel game building toolkit.

hello blocks

Following the “hello world” example at voxeljs.com, we generate a simple Minecraft-like blocks world in which we can walk around and dig (you can visit it here). The example gives us a JavaScript file, builtgame.js, that we can also use from Caffeine.

As generated, builtgame.js evaluates its createGame function at load time. This creates an HTML5 canvas, initializes WebGL, and begins the game when the hosting page is loaded. We want to save those steps for SqueakJS to initiate, and we also want to use a canvas of our own, in a Caffeine window.

hooks for Caffeine

We can achieve the first part by changing builtgame.js so that it just puts createGame somewhere SqueakJS can get to it, instead of evaluating it. We can create a property on the browser DOM window for this:

window.game = createGame;

Normally we would edit the source projects from which builtgame.js is generated, rather than builtgame.js directly (properly forking the corresponding repositories), but for this example we’ll just go ahead.

Voxel.js uses the three.js framework as its WebGL interface. The three.js WebGL renderer accepts an HTML5 canvas parameter for its initialization function. The second change we’ll make to builtgame.js is to pass a canvas set by SqueakJS in another window property:

View.prototype.createRenderer = function() {
  this.renderer = new THREE.WebGLRenderer({
    canvas: window.gameCanvas !== undefined ? window.gameCanvas : undefined,
    antialias: true});
  this.renderer.setSize(this.width, this.height);
  this.renderer.setClearColorHex(this.skyColor, 1.0);
  this.renderer.clear();};

Finally, we’ll change the game rendering initialization function, to save a reference to the voxel.js renderer’s event emitter, so that we can tell it to pause from SqueakJS:

Game.prototype.initializeRendering = function(opts) {
  var self = this;
  if (!opts.statsDisabled) self.addStats();
  window.addEventListener('resize', self.onWindowResize.bind(self), false);
  self.ee = (
    requestAnimationFrame(window).on(
      'data',
      function(dt) {
        self.emit('prerender', dt);
        self.render(dt);
        self.emit('postrender', dt);
      });
  if (typeof stats !== 'undefined') {
    self.on(
      'postrender',
      function() {
        stats.update();});}}

on the Smalltalk side

Now, in SqueakJS, we can create a VoxelJS class:

Object
  subclass: #VoxelJS
  instanceVariableNames: ''
  classVariableNames: ''
  poolDictionaries: ''
  category: 'Hex-HTML5-WebGL-VoxelJS'.

VoxelJS class instanceVariableNames: 'game'

We’ll give it a class-side method to load and initialize voxel.js, with a Caffeine window and canvas for it to use:

initialize

| canvas |

Webpage current loadScriptFrom: 'js/voxeljs/builtgame.js'.
canvas := Webpage createWorldOfKind: 'voxeljs'.
canvas styleAt: #borderRadius put: '10px'.

(Webpage current)
  windowizeElementNamed: canvas window id
  closingWith: [
    self pause.
    Webpage current top at: #gameCanvas put: nil].

canvas window dragWith: canvas window windowButtonsTray moveButton.
game := (
  (Webpage current top)
    at: #gameCanvas put: canvas;
    game: {#container -> canvas window})

We’ll also add a method for pausing the voxel.js renderer, using the ee property we added to the game rendering initialization in builtgame.js:

pause
  game ee pause.
  (JQuery at: #fps) element remove

In a workspace, we send our initialization message:

VoxelJS initialize

Now we have our first voxel world, running in a Caffeine window that we can easily close, rather than the whole screen. If you clear your browser cache (including IndexedDB) for caffeine.js.org, you can reload the Caffeine page to see this code in action.

Please let me know if you get this far!

 

toward a virtual machine browser

Posted in Naiad, Smalltalk, Spoon with tags , , on 27 November 2013 by Craig Latta

I just had an idea for the design of a “virtual machine browser”, for browsing the internal state of a running virtual machine simulator and the objects of the memory it’s running. Something more sophisticated than printing the debug support messages to the Transcript.

%d bloggers like this: