Generating adversarial images
in the browser with tensorflow.js

Live demo + full code listing + some background on memory management and asynchronous execution in tensorflow.js. (April 2021, about the author)

Introduction

In this tutorial, I'll demonstrate how to create adversarial images (i.e. images that fool neural networks into giving wrong classifications) using the JavaScript library tensorflow.js.

For this tutorial I chose the Fast Gradient Sign Method — one of the first published methods for generating adversarial images and one that is pretty straightforward to implement. Nonetheless, writing it in tensorflow.js harbours some pitfalls that you might not be expecting, in particular if you are used to writing machine learning code in python.

Here is what will be covered in the article:

A live demo that generates an adversarial image in your browser
Targeted Fast Gradient Sign method in tensorflow.js
Pitfall: Memory management
Pitfall: Asynchronous execution
Full code listing

This tutorial is not an introduction to adversarial images, so if you aren't familiar yet with the topic, you might want to read this blog post first. If you are not interested in implementing adversarial attacks yourself, but are looking for a JavaScript library that implements several different methods, then adversarial.js by kennysong has got you covered.

Live demo

If you press the "Go" button in the demo below, your browser will execute a script that:

Downloads the mobilenet classification model
Classifies the original image of the flower and displays the result
Runs the attack on the original image and displays the resulting adversarial image
Classifies the adversarial image and displays the result

This demo (at the time of writing) works best in Chrome and should just take a few seconds to run. In other browsers it might take longer than that [1]. There is also a standalone version of the demo, which you can find here.

If you look closely, you will see some noisy artifacts in the generated adversarial image. This noise (called perturbation) does not fool humans (it is obviously still a flower), but is what fools the neural network into classifying the image as a vase.

It is entirely possible to generate adversarial images without any humanly visible noise artifacts that still fool the neural network. The reason you can see the artifacts here is that I chose to set the parameter of the attack strength epsilon to a pretty high ε=5, meaning the attack is allowed to change any pixel value in the image by ±5.

Now let's dig down into how the adversarial image is being generated! If you just want to see the code, you can skip down to the end of the page.

Targeted Fast Gradient Sign method in tensorflow.js

Generating an adversarial is pretty intuitive: to "turn" an image into some target class, we need to change the pixel values (= apply a perturbation) so that the altered image is nearer to the target class than to the original class of the image.

In the Targeted Fast Gradient Sign method, we first calculate the loss J between model output for the original image and the target class. We are then interested in the gradient of that loss with respect to the pixel values of the original image (upside-down triangle), i.e. how would we need to change the pixel values of the original image, so that the loss decreases:

$SVG containing the formula for the targeted Fast Gradient Sign Method: adversarial = original - ε × sign(gradient_{original}(loss(original, targetClass)))$

A bit tricky to figure out is how to calculate the gradient with respect to an image in tensorflow.js:

    
// class ID for vase, in total there are 1000 classes 
const targetClass = 883;

// tf.grad expects a loss function that takes only one input: one (image) tensor
// however, we want the cross-entropy loss with respect to an image AND the target class
// we get around this by specifying the target class outside the loss function
// that way it is still in scope within the loss function
function loss(image){
   let targetOneHot = tf.oneHot([targetClass], 1000);
   let logits = model.infer(image);
   return tf.losses.softmaxCrossEntropy(targetOneHot, logits);
}

// now we can initialise a function that calculates a gradient
// given the specified loss function
const calculateGradient = tf.grad(loss);

Once we have the gradient function defined, we can directly implement the attack using the formula from above:

        
// read the original image from the website into a tensorflow.js tensor 
const originalImage = document.getElementById("original-image");
const originalTensor = tf.browser.fromPixels(originalImage);

// calculate the adversarial
const gradient = calculateGradient(originalTensor);
const perturbation = gradient.sign().mul(epsilon);
let adversarial = tf.sub(originalImage, perturbation);

// pixel values must be between [0, 255]
adversarial = adversarial.clipByValue(0, 255);

Pitfall: Memory management

Usually when programming in JavaScript (or Python), there is no need to care at all about memory management. The runtime allocates memory as needed (e.g. when initializing a new variable) and a garbage collector takes care of freeing that memory again when it is no longer needed.

If you allocate a tensor in tensorflow.js, however, this does not happen automatically. Instead, tensorflow.js will keep the tensor for you until you tell it to let go of it. For example, let's try creating new tensors in a loop and printing info on how much main memory is allocated by tensorflow.js:

        
const originalElement = document.getElementById("original-image");
let originalTensor = null;

for (let i=0;i<=1000; i++){
   console.log(tf.memory().numBytes + " Bytes allocated at i=" + i);
   originalTensor = tf.browser.fromPixels(originalElement);
}

If you try running this code snippet, you will see that your browser will have allocated 612 Megabyte of main memory, even though you are "overwriting" the variable during every run of the loop:

        
Outputs:
> 0 Bytes allocated at i=0
> 602112 Bytes allocated at i=1
> 1204224 Bytes allocated at i=2
> 1806336 Bytes allocated at i=3
   ...
> 602112000 Bytes allocated at i=1000

Of course we can't go around creating such memory leaks and using up all our users' memory. At some point they might even get a warning from their browser that there is a website hogging all their memory, urging them to stop your script. Embarrassing.

a) Manually disposing tensors

The first way to fix the situation is to let tensorflow.js know that you no longer need a tensor by calling the .dispose() method on the tensor:

        
const originalElement = document.getElementById("original-image");
let originalTensor = null;

for (let i=0;i<=1000; i++){
   console.log(tf.memory().numBytes + " Bytes allocated at i=" + i);
   originalTensor = tf.browser.fromPixels(originalElement);

   originalTensor.dispose(); // <- de-allocating the memory
}

        
Outputs:
> 0 Bytes allocated at i=0
> 0 Bytes allocated at i=1
> 0 Bytes allocated at i=2
> 0 Bytes allocated at i=3
   ...
> 0 Bytes allocated at i=1000

This can become a bit cumbersome, in particular if you want to do some inline method calls:

        
const originalElement = document.getElementById("original-image");

// memory leak because .div creates a new tensor we are not disposing
let originalTensor = tf.browser.fromPixels(originalElement);
console.log(originalTensor.div(255));
originalTensor.dispose();

// instead, need to store the normalized tensor in a variable so we can dispose it
let originalTensor = tf.browser.fromPixels(originalElement);
let originalTensorNormalized = originalTensor.div(255);
console.log(originalTensorNormalized);
originalTensor.dispose();
originalTensorNormalized.dispose();

b) Disposing tensors using tf.tidy

Alternatively to manually disposing all tensors, you can also wrap your code in a tf.tidy function call:

        
function logImage(){
   const originalElement = document.getElementById("original-image");
   let originalTensor = tf.browser.fromPixels(originalElement);
   console.log(originalTensor.div(255));
   return "some return value";
}

// tf.tidy disposes the originalTensor as well as the tensor created by .div
let result = tf.tidy(logImage);

Let's take a closer look at what happens during the call to tf.tidy (if you know context managers in Python, the following will seem quite familiar to you):

        
// based on https://github.com/tensorflow/tfjs/blob/tfjs-v3.2.0/tfjs-core/src/engine.ts#L475
startScope(); // starts tracking tensor creation
try {
   const res = logImage();
   endScope();  // dispose all the tracked tensors
   return res;
}
catch ex(){
   endScope(); // dispose all the tracked tensors in case of error
   throw ex;
}

Of course, we usually don't bother with defining a proper named function, but instead call tf.tidy with an anonymous function:

        
let result = tf.tidy(() => {
    const originalElement = document.getElementById("original-image");
    let originalTensor = tf.browser.fromPixels(originalElement);
    console.log(originalTensor.div(255));
    return "some return value";
});

Pitfall: Asynchronous execution

In the live demo, the generation of the adversarial takes a few seconds or more to run. The problem is that JavaScript is generally single-threaded — long-running calculations like ours could, in principle, lock up that thread and prohibit the user from further interacting with the website while the calculations are running

To avoid that from happening, tensorflow.js is implemented for asynchronous execution:

  
const originalElement = document.getElementById("original-image");

// classification is a long-running task, but is implemented asynchronously
// this means that the call to model.classify returns immediately
// but returns not the classification results but a special object called "Promise""
// (outputs: Promise{<pending>})
console.log(model.classify(originalElement));

// if we want to get the actual classification result, we use "await" to signal
// that we depend on this result and to resume execution here when it becomes available
const originalPredictions = await model.classify(originalElement);

// (outputs: [{className: "daisy", probability: 0.9708568453788757}...]
console.log(originalPredictions);

Be aware that you only need to take care of using await when you are moving data from "tensorland" to native JavaScript types. You don't need to await if you are doing operations on tensors that return other tensors:

        
// returns a tensor object
const gradient = calculateGradient(originalImage);

// trying to get the underlying data gives us a Promise
// (outputs: Promise{<pending>})
console.log(gradient.data());

// but we can perform further operations on the tensor object without
// having to use any "await"
const perturbation = gradient.sign().mul(epsilon);
let adversarial = tf.sub(originalImage, perturbation);

// getting the underlying data of result again gives us a Promise
// (outputs: Promise{<pending>})
console.log(adversarial.data())

// two options for getting the actual adversarial image
adversarial = await adversarial.data(); // option1: you do the await
adversarial = adversarial.dataSync();   // option2: tensorflow.js does the await

If you are interested, you can read more about asynchronous operations in JavaScript here.

Full code listing

Now let's put all the bits together! The code listing below contains all the Javascript you need to generate your own adversarial images. There are copious amounts of comments in the code, so it hopefully is easy to follow along.

You will need tensorflow.js (code is tested with version 3.2.0) and mobilenet (tested with library version 2.1.0). Everything else is vanilla Javascript, no other libraries are needed.

You can find the HTML and CSS used in the live demo on my github. There is also a standalone version of the demo, which you can find here.

        
// FGSM configuration; these values are cherry-picked
// (FGSM is the not strongest attack, and the selected flower mostly
//  turns into a vase, no matter what targetClass was selected)
const targetClass = 883; // class ID for vase
const epsilon = 5.0;   // strength of the perturbation

// loaded model will be stored here
let model = null;


// just a simple helper function that takes a prediction dictionary from tensorflow.js
// and formats it into a nice string
function formatPrediction(prediction){
    const roundedProbability = Math.round(prediction["probability"] * 100.0) / 100.0;
    return prediction["className"] +" (" + roundedProbability + ")";
}


// perform a targeted Fast Gradient Sign method attack
// trying to "turn" the original image into the given target class
function targetedFGSM(model, originalImage, targetClass, epsilon){
    // tf.grad expects a loss function that takes only one input: one (image) tensor
    // however, we want the cross-entropy loss with respect to an image AND the target class
    // we get around by specifying the loss function inside the outer function (closure)
    // this way the function has access to both the image as well as the target class
    function loss(image){
        let targetOneHot = tf.oneHot([targetClass], 1000);
        let logits = model.infer(image);
        return tf.losses.softmaxCrossEntropy(targetOneHot, logits);
    }

    // now we can initialise a function that calculates a gradient
    // given the specified loss function
    const calculateGradient = tf.grad(loss);

    // tf.tidy automatically disposes all tensors that are not needed anymore
    // (so we don't get memory leaks)
    return tf.tidy(() => {
            // let's get the gradient with respect to the original image
            const gradient = calculateGradient(originalImage);

            // apply the fast gradient sign method
            const perturbation = gradient.sign().mul(epsilon);
            let adversarial = tf.sub(originalImage, perturbation);

            // pixel values must be between [0, 255]
            adversarial = adversarial.clipByValue(0, 255);

            return adversarial;
        });
}


// this function is called when the "Go!" button is clicked
// any function that "awaits" asynchronous functions, must itself be marked as async
async function runAttack(){
    // if this is the first time the button was clicked, we need to load the model
    if (model === null)
        model = await mobilenet.load();

    // how much memory is tensorflow.js using before generating the adversarial image?
    const initialMemoryUsage = tf.memory().numBytes;

    // run the classifier on the original image
    // the result is an array with the Top3 predictions
    const originalElement = document.getElementById("original-image");
    const originalPredictions = await model.classify(originalElement);

    // lets write the highest-probable prediction onto the webpage
    const originalTextElement = document.getElementById("original-text");
    originalTextElement.innerHTML = formatPrediction(originalPredictions[0]);

    // to generate the adversarial,
    // we let tensorflow grab the image data from the  DOM element
    // and then run the targetedFGSM function
    const originalTensor = tf.browser.fromPixels(originalElement);
    const adversarialTensor = targetedFGSM(model, originalTensor, targetClass, epsilon);

    // display the adversarial image on the webpage
    // need to store the normalized tensor into a variable
    // so we can dispose it later (avoid memory leaks)
    const adversarialElement = document.getElementById("adversarial-image")
    const adversarialTensorNormalized = adversarialTensor.div(255);
    tf.browser.toPixels(adversarialTensorNormalized, adversarialElement);

    // run the classifier on the generated adversarial image
    const adversarialPredictions = await model.classify(adversarialTensor);

    // and again write the highest-probable prediction onto the webpage
    const adversarialTextElement = document.getElementById("adversarial-text");
    adversarialTextElement.innerHTML = formatPrediction(adversarialPredictions[0]);

    // clean up to avoid memory leaks
    originalTensor.dispose();
    adversarialTensor.dispose();
    adversarialTensorNormalized.dispose();

    // check if still have any memory leaks
    const leakingMemory = tf.memory().numBytes - initialMemoryUsage;
    console.log("Memory leakage: " + leakingMemory + " bytes");
}

Footnotes

[1] tensorflow.js uses WebGL (Web Graphics Library) for GPU acceleration, where possible. It seems that at the time of writing this works only in Chrome, on other browsers tensorflow.js is running without GPU acceleration. [back]

About the author

Generating adversarial images in the browser with tensorflow.js