Archive for debugging

Exploring the Netflix player with the Caffeine Chrome extension

Posted in Caffeine, consulting, Context, livecoding, music, Smalltalk, SqueakJS with tags , , , , , , , , , , on 22 September 2019 by Craig Latta
debugging Better Call Saul with Caffeine

With the latest version of the Caffeine Chrome extension, you can run Caffeine in a Chrome DevTools panel, with access to all the Chrome debugging APIs. I’ve been using it to explore the Netflix video player, for an app I’m writing that enables the viewer to edit narratives by rearranging scenes.

From a quick look at the DOM element tree for the player, it’s apparent that it’s a React app. By following a reference chain from a user interface element (like the skip-forward button), through the bound “this” object of its click-event listener, I found the internal React properties for all the player’s UI elements, and all the player functions they use (for example, for seeking forward in a video).

With those functions in hand, I made a Netflix player class in Smalltalk, which can manipulate the Netflix player React app interactively from Smalltalk code. Other objects I made representing show elements (like scenes, episodes, seasons, and series) can use my player to compile analytic information about shows, and present them in different ways. For example, you could watch an episode of Better Call Saul consisting only of scenes that include a certain character, or that take place at a certain location, or with flashbacks placed in chronological order. This is for a webapp I’m writing called Arc.

I’m eager to see what else you explore using the Caffeine extension in the DevTools!

Livecoding other tabs with the Chrome Remote Debugging Protocol

Posted in Appsterdam, consulting, Context, Smalltalk, SqueakJS with tags , , , , , , , on 24 July 2017 by Craig Latta

Chrome Debugging Protocol

We’ve seen how to use Caffeine to livecode the webpage in which we’re running. With its support for the Chrome Remote Debugging Protocol (CRDP), we can also use it to livecode every other page loaded in the web browser.

Some Help From the Inside

To make this work, we need to coordinate with the Chrome runtime engine. For CRDP, there are two ways of doing this. One is to communicate using a WebSocket connection; I wrote about this last year. This is useful when the CRDP client and target pages are running in two different web browsers (possibly on two different machines), but with the downside of starting the target web browser in a special way (so that it starts a conventional webserver).

The other way, possible when both the CRDP client and target pages are in the same web browser, is to use a Chrome extension. The extension can communicate with the client page over an internal port object, created by the chrome.runtime API, and expose the CRDP APIs. The web browser need not be started in a special way, it just needs to have the extension installed. I’ve published a Caffeine Helper extension, available on the Chrome Web Store. Once installed, the extension coordinates communication between Caffeine and the CRDP.

Attaching to a Tab

In Caffeine, we create a connection to the extension by creating an instance of CaffeineExtension:

CaffeineExtension new inspect

As far as Chrome is concerned, Caffeine is now a debugger, just like the built-in DevTools. (In fact, the DevTools do what they do by using the very same CRDP APIs; they’re just another JavaScript application, like Caffeine is.) Let’s open a webpage in another tab, for us to manipulate. The Google homepage makes for a familiar example. We can attach to it, from the inspector we just opened, by evaluating:

self attachToTabWithTitle: 'Google'

Changing Feelings

Now let’s change something on the page. We’ll change the text of the “I’m Feeling Lucky” button. We can get a reference to it with:

tabs onlyOne find: 'Feeling'

When we attached to the tab, the tabs instance variable of our CaffeineExtension object got an instance of ChromeTab added to it. ChromeTabs provide a unified message interface to all the CRDP APIs, also known as domains. The DOM domain has a search function, which we can use to find the “I’m Feeling Lucky” button. The CaffeineExtension>>find: method which uses that function answers a collection of search results objects. Each search result object is a proxy for a JavaScript DOM object in the Google page, an instance of the ChromeRemoteObject class.

In the picture above, you can see an inspector on a ChromeRemoteObject corresponding to the “I’m Feeling Lucky” button, an HTMLInputElement DOM object. Like the JSObjectProxies we use to communicate with JavaScript objects in our own page, ChromeRemoteObjects support normal Smalltalk messaging, making the JavaScript DOM objects in our attached page seem like local Smalltalk objects. We only need to know which messages to send. In this case, we send the messages of HTMLInputElement.

As with the JavaScript objects of our own page, instead of having to look up external documentation for messages, we can use subclasses of JSObject to document them. In this case, we can use an instance of the JSObject subclass HTMLInputElement. Its proxy instance variable will be our ChromeRemoteObject instead of a JSObjectProxy.

For the first message to our remote HTMLInputElement, we’ll change the button label text, by changing the element’s value property:

self at: #value put: 'I''m Feeling Happy'

The Potential for Dynamic Web Development

The change we made happens immediately, just as if we had done it from the Chrome DevTools console. We’re taking advantage of JavaScript’s inherent livecoding nature, from an environment which can be much more comfortable and powerful than DevTools. The form of web applications need not be static files, although that’s a convenient intermediate form for webservers to deliver. With generalized messaging connectivity to the DOM of every page in a web browser, and with other web browsers, we have a far more powerful editing medium. Web applications are dynamic media when people are using them, and they can be that way when we develop them, too.

What shall we do next?

 

Caffeine: live web debugging with SqueakJS

Posted in Appsterdam, consulting, Context, Naiad, Smalltalk, Spoon with tags , , , , , , , , , , , , , , , , , , , , on 26 October 2016 by Craig Latta

In February 2015 I spoke about Bert Freudenberg’s SqueakJS at FOSDEM. We were all intrigued with the potential of this system to change both Smalltalk and web programming. This year I’ve had some time to pursue that potential, and the results so far are pretty exciting.

SqueakJS is a Squeak virtual machine implemented with pure JavaScript. It runs in all the web browsers, and features a bi-directional JavaScript bridge. You can invoke JavaScript functions from Smalltalk code, and pass Smalltalk blocks for JavaScript code to invoke as callbacks. This lets Smalltalk programmers take advantage of the myriad JavaScript frameworks available, as well as the extensive APIs exposed by the browsers themselves.

The most familiar built-in browser behavior is for manipulating the structure of rendered webpages (the Document Object Model, or “DOM”). Equally important is behavior for manipulating the operation of the browser itself. The Chrome Debugging Protocol is a set of JavaScript APIs for controlling every aspect of a web browser, over a WebSocket. The developer tools built into the Chrome browser are implemented using these APIs, and it’s likely that other browsers will follow.

Using the JavaScript bridge and the Chrome Debugging Protocol, I have SqueakJS controlling the web browser running it. SqueakJS can get a list of all the browser’s tabs, and control the execution of each tab, just like the built-in devtools can. Now we can use Squeak’s user interface for debugging and building webpages. We can have persistent inspectors on particular DOM elements, rather than having only the REPL console of the built-in tools. We can build DOM structures as Smalltalk object graphs, complete with scripted behavior.

I am also integrating my previous WebDAV work, so that webpages are manifested as virtual filesystems, and can be manipulated with traditional text editors and other file-oriented tools. I call this a metaphorical filesystem. It extends the livecoding ability of Smalltalk and JavaScript to the proverbial “favorite text editor”.

This all comes together in a project I call Caffeine. had fun demoing it at ESUG 2016 in Prague. Video to come…

new website for Black Page Digital

Posted in Appsterdam, consulting, Context, GLASS, music, Naiad, Seaside, Smalltalk, Spoon with tags , , , , , , , , , , , , , , , , on 21 January 2016 by Craig Latta

I wrote a new website for Black Page Digital, my consultancy in Amsterdam and San Francisco. It features a running Squeak Smalltalk that you can use for livecoding. Please check it out, pass it on, and let me know what you think!pano

Context status 2015-01-16

Posted in Appsterdam, consulting, Context, Naiad, Smalltalk, Spoon with tags , , , , , , , on 16 January 2015 by Craig Latta

Hoi all–

Context is the umbrella project for Naiad (a distributed module system for all Smalltalks), Spoon (a minimal object memory that provides the starting point for Naiad), and Lightning (a remote-messaging framework which performs live serialization, used by Naiad for moving methods and other objects between systems). I intend for it to be a future release of Squeak, and a launcher and module system for all the other Smalltalks. I’m writing Context apps for cloud computing, web services, and distributed computation.

Commits b7676ba2cc and later of the Context git repo have:

  • Support for installable object memories as git submodule repos.
  • Submodule repos for memories for each of the known Smalltalk dialects, with Naiad support pre-loaded. I’m currently working on the submodules for Squeak and Pharo.
  • A web-browser-based console for launching and managing object memories.
  • A WebDAV-based virtual filesystem that enables Smalltalk to appear as a network-attached storage device, and mappings of the system to that filesystem that make Smalltalk accessible from external text editors (e.g., for editing code, managing processes and object memories).
  • Remote code and process browsers.

There’s a live discussion site and a mailing list. The newsgroup is gmane.comp.lang.smalltalk.squeak.context.

Thanks for checking it out!

Craig

new Context active filesystem layout

Posted in Appsterdam, consulting, Context, Naiad, Smalltalk, Spoon with tags , , , , , , , , , , , , , on 22 December 2014 by Craig Latta

When you start the Context app, you start a webserver that provides a “console”. Viewed through a host web browser, the console describes what Context is, and enables control of the memories it knows about. The webserver also provides an active filesystem via WebDAV. This lets you interact with the console from a host terminal or text editor, in a manner reminiscent of a Unix procfs (content is generated live-on-read). Here’s a typical filesystem layout, and what you can do with it:

/
   README.html

   memories
      3EAD9A45-F65F-445F-89C1-4CA0A9D5C2F8
         session
            state
            performance
         classes
            Object
               metaclass
                  (etc.)
               methods
                  at:
                  (etc.)
               slots
                  all
                     (etc.)
                  inherited
                     (etc.)
                  local
                     (etc.)
               subclasses
                  (etc.)
         processes
            the idle process
               ProcessorScheduler class>>idleProcess
                  source
                  variables
                     thisContext
                     self
                     (etc.)
               [] in ProcessorScheduler class>>resume
               (etc.)
            (etc.)
         workspaces
            hello world
               source
               result
                  7

The README.html file is what the console displays initially. It has a directory sibling memories, containing a subdirectory for each memory the console knows about. Each memory is named by its UUID. In the session directory, there are files which give information about a memory. The state file looks like this:

# This memory is running. You can send it one of the following
# commands: snapshot, suspend, or stop. To do so, write this file with
# the desired command as the first word after this comment. Subsequent
# comments give other information about this memory, like host
# resource usage and virtual machine plugins loaded.

(type command here)

# host resource usage
#
# bytes used:        437,598
# bytes available: 1,328,467

# virtual machine plugins loaded
#
# FlowPlugin

In this way, a file in the active filesystem provides access to a read-eval-print loop (REPL). The user gives input to the console by writing the file; the console gives feedback to the user (including documentation) by generating appropriate content when the file is read.

The performance file looks like this:

# instructions per second: 382,184,269
# messages per second:      12,355,810

This gives general profiling information about the virtual machine.

The subdirectories of the classes directory correspond to the memory’s classes. Each one has subdirectories for its methods, subclasses, and metaclass. The methods directory has a file for each method of the class. This provides the ability to browse and change source code in the memory from a host text editor.

The processes directory has a subdirectory for each running process in the memory. Each process directory has a subdirectory for each context of that process. Each context directory has a REPL file for the source code of the context’s method, and a subdirectory for the context’s variables (including the context itself), each of which is an inspector in the form of a REPL file. In this way, much of the functionality of the traditional Smalltalk debugger is accessible from a host text editor.

Finally, the workspaces directory has subdirectories for multiple “workspaces”, where one may evaluate expressions and interact with their result objects. Each workspace has a source file, another REPL file which contains instructions, the expression to evaluate, and, on the next read after write, the textual form of the result. In addition, in a result directory, is a file named for the textual form of the result, containing a REPL inspector for that result object.

These tools are useful both for newcomers to live object systems who are more comfortable with a text editor than the Smalltalk GUI, and for those accessing systems running in the cloud, for which traditional GUI access might be awkward or prohibitive.

Smalltalk Reflections episode three is up

Posted in Appsterdam, consulting, Context, music, Smalltalk, Spoon with tags , , , , , , , , , , , , , , on 16 December 2014 by Craig Latta

Check it out!

a most useful virtual machine debugging aid: simulated objects

Posted in Appsterdam, consulting, Context, Smalltalk, Spoon with tags , , , , , , on 2 December 2014 by Craig Latta

Squeak’s virtual machine simulator is extremely useful for debugging. You can use it to inspect and change objects “while time is stopped”, between the execution of individual virtual machine instructions. Traditionally, though, it takes an address-based view of objects. There are several useful utility methods which, given an object address, will print useful information to the Transcript. Wouldn’t it be nicer, though, if you could use normal inspectors to look through the fields of the objects in a simulated virtual machine’s object memory?

I created simulated objects for this purpose. They are instances of a SimulatedObject class; each one has an interpreter and an address. They can print useful information about themselves, like the interpreter can, but they can also modify themselves and interact with each other, changing the interpreter’s object memory appropriately. Are you wondering about the instructions of a compiled method? Would you like to make a few choice modifications to those instructions? A simulated object for that method’s address will help you.

Simulated objects play nicely with Squeak’s object inspectors, and, more importantly, with its object explorers. You feel like you’re inspecting normal objects, except that you can’t send normal messages to them. Or can you? I’m pondering this. It might be useful, for example, to terminate a process in a simulated interpreter’s object memory, without having to do it in another process. Time is stopped, but perhaps you could queue up messages to send when it starts again, through a collaboration between simulated objects and a coordinating object in the memory they describe.

I’ve been using simulated objects recently to chase references with the absolute assurance that I won’t be creating new ones. They’re very useful for debugging virtual machine primitives. Sometimes, when I’m debugging a headless system with a broken remote messaging system, it’s the only user interface I have for inspecting things. And it’s sure a lot nicer than inspecting things in a C debugger.

What will you do with them?

debugging remote exceptions works

Posted in consulting, Context, Smalltalk, Spoon with tags , , , , , , on 20 November 2014 by Craig Latta

a debugger for a remote unhandled exception

a debugger for a remote unhandled exception

I have debugging working for remote unhandled exceptions. My motivating use case was debugging messages not understood by the Context console’s embedded web server. The console is a headless app. In development, I run it with a remote-messaging connection to a headful system. Now, when there is an unhandled exception (like a message not understood), the exception requests that the headful system open a debugger (as its default action).

Before opening the debugger, the headful system replaces the sender of the first relevant context on the headless system with the last relevant context on the headful system, hiding all the remote-messaging-related contexts in between. The picture above shows an example of this. On the headful system, I sent “zork” to an object on the headless system. The debugger shows a continuous context stack which spans the two systems. This all works with little special handling in the debugger because of the complete transparency of remote messaging. It doesn’t matter that the contexts and methods that the debugger is manipulating happen to be remote.

Another VM debugging trick: halting

Posted in consulting, Smalltalk, Spoon with tags , , , , on 31 December 2013 by Craig Latta

Just another little VM debugging trick I use… In one of my VM plugins I include a little C function called “breakHere”, which Smalltalk can call as a primitive. I put a breakpoint on it in the C debugger, and then have a convenient way to break in the C debugger from Smalltalk.

%d bloggers like this: