/

Ghost in the Mac

The gif below shows something I thought would be very hard to do with JavaScript: a ghost (image) is floating across browser windows.

Ghost in the Mac
Watch the ghost travel across browser windows

Don’t think the gif is surprising? Just using websockets, right? Sure, using websockets you can have windows communicate with each other, but here the windows are orchestrated in a way that the position of the window on the screen impacts whether or not you can see the ghost. It looks so seamless, it’s easy to forget that this isn’t how you’d expect browser windows to behave.

Github Link to working code

Confusing?

Watch this YouTube video I made explaining how this works.

What I used to build this

  1. A Mac. This same experience could be done on Linux, but the mechanism would need to be different, as you’ll see.
  2. A JavaScript web server. I’m using Bun but you could use Node or Deno. It’s important (well, sort of) that the web server runs JavaScript because you will be using JXA.
  3. JXA (JavaScript for Automation). It’s a scripting language introduced by Apple to provide macOS users with a way to automate tasks using JavaScript. Never heard of it, right? Well, it’s on your Mac right now. We’ll get into what it is soon.
  4. Google Chrome. Any web browser would probably do. Chrome worked for me.

What on earth is JXA?

From this link:

JXA is one of those things that Apple designed broken–and then apparently left to rot.

JXA is a way of using JavaScript to control your Mac. It’s very weird and very hard to understand. I say this as someone who’s pretty comfortable with JavaScript. It doesn’t behave how you think it would.

Here’s some annotated JXA code. It looks kinda like the JS you are used to, but it is slippery.

// Sometimes JXA requires you to import a bridge to Objective C
// I'm not sure why.
ObjC.import("AppKit")

// This is normal JS code.
const CLIENT_WINDOW_COUNT = 3
const GAP = 100

// The $ comes from the objective c bridge. It was the only way to get the screen resolution.
const mainScreen = $.NSScreen.mainScreen
const screenFrame = mainScreen.frame
const screenWidth = screenFrame.size.width
const screenHeight = screenFrame.size.height

// more regular JavaScript
const windowWidth =
  (screenWidth - (GAP * CLIENT_WINDOW_COUNT - 1)) / CLIENT_WINDOW_COUNT
const windowHeight = screenHeight / 2

// put Google Chrome in a variable
const chrome = Application("Google Chrome")
for (let i = 0; i < CLIENT_WINDOW_COUNT; i++) {
  // Google Chrome allows you to make a window.
  // Not all applications allow for this, but Chrome does.
  const win = chrome.Window().make()

  // Tell the window to visit this url
  win.activeTab.url = "http://localhost:3000/" + win.id()
  const x = i * windowWidth + i * GAP
  win.bounds = {
    x: x,
    y: windowHeight / 2,
    width: windowWidth,
    height: windowHeight,
  }
}

So, that code let’s me create Chrome windows and navigate to a webpage. Very cool. Very difficult to debug.

Why is JXA so weird?

JXA is weird because it’s the JavaScript equivalent of another Language: AppleScript. AppleScript is a 31 year old (as of 2024) language.

This is AppleScript. I promise you. It’s real.

try
tell application "Finder" to set the this_folder
    to (folder of the front window) as alias

on error -- no open folder windows
set the this_folder to path to desktop folder as alias
end try

set thefilename to text returned of (display dialog
"Create file named:" default answer "filename.txt")
set thefullpath to POSIX path of this_folder & thefilename
do shell script "touch \"" & thefullpath & "\""

So, in 2014 JXA was released as way of doing the above in JavaScript. Apparently, JXA is no longer being updated. It’s a mess.

So, Bun runs the JXA code?

No. A program called osascript runs the JXA code.

Here’s an example how how this work:

// this function is run by Bun.
async function fetchWindowPositions() {
  const res: Array<ClientWindow> = await run(
    // this function, inside run, is not run by Bun.
    // It's run by osascript.
    // the `run` function takes this function and calls toString() on it
    // so, this function cannot reference the code outside
    () => {
      // We return values which are emitted in the Promise
      return Application("Google Chrome")
        .windows()
        .map((win: any): ClientWindow => {
          const bounds = win.bounds()
          return {
            id: win.id(),
            x: bounds.x,
            y: bounds.y,
            width: bounds.width,
            height: bounds.height,
          }
        })
    }
  )

  clientWindows.length = 0

  // we finally use the results here
  res.forEach((v) => clientWindows.push(v))
}

So, how does the ghost move across browsers?

Roughly…

  1. The Bun application boots up.
  2. It determines the size of the screen using JXA.
  3. It creates a ghost that moves around every so often (setInterval).
  4. It launches 3 Chrome browsers (JXA). Each browser navigates to the Bun server and sends their unique window id. They keep pining the Bun Server.
  5. It monitors the positions of the Chrome windows (JXA).
  6. The Bun server tells the clients where the ghost is relative to them.

If you are curious, please check the Github Link to working code.