TensorFlow.js for React Native: Up & Running!

Earlier this month, TensorFlow.js for React Native was made generally available following an alpha release phase.

React Native

React Native is a popular framework for mobile app development. It utilises native components for optimal performance, but the logic which controls those components is written in JavaScript using the React front-end framework. Although React Native does not use HTML and CSS, it uses a similar syntax to define and style native components, making mobile app development feasible for those already familiar with web technologies.

TensorFlow.js

TensorFlow.js is the leading machine learning library for JavaScript. It previously ran in both web browser environments, and on Node.js.

The core of TensorFlow.js is not dependent on any particular JavaScript environment.

Why then, do we need a TensorFlow.js package for React Native support?

TensorFlow.js should, at least in theory, already have worked in a new environment such as React Native. However, it would be limited to the pure JavaScript, CPU-based back-end implementation, which is functional but does not provide optimal performance.

So the React Native platform adapter for TensorFlow.js adds a new backend, rn-webgl, which is based on WebGL, similar to the WebGL backend that we use when we run TensorFlow.js in a web browser. This leads to significantly better performance (as explained in detail in my course, Machine Learning in JavaScript with TensorFlow.js). React Native does not have access to WebGL in the same way that web browsers do, so the standard WebGL backend would not work. Instead, the new React Native WebGL backend is implemented using the Expo GL library (which implements the WebGL specification via OpenGL ES). This provides similar performance to the browser implementation of TensorFlow.js – the main influencing factor being the graphics power available on the device.

The React Native platform adapter also brings additional utilities designed to make it easier to use in the React Native environment. This includes new IO Handlers for loading and saving models, and image and video handling (for example to use video from the phone’s camera).

Setting up a React Native project for TensorFlow.js

The official documentation provides instructions for installing and configuring TensorFlow.js in a React Native app.

The Bad News

Unfortunately, at the time of writing, there are six steps required to configure TensorFlow.js in React Native, and many of these steps involve also reviewing the documentation for dependencies which need to be installed in order for TensorFlow.js to work. The configuration process is particularly complex if you choose to use the React Native CLI rather than Expo for your project.

If you are adding TensorFlow.js to an existing project, then you’ll need to follow those configuration steps, there’s no getting around it. At least in this case, you probably have enough experience with React Native to work through the required configuration steps.

The Good News

If you simply want to get started and start playing with TensorFlow.js in a new project, then I’ve already done the configuration for you!

You can use my React Native template with TensorFlow.js support to get up and running with a single command (so long as you have npm installed).

Just run this command in the terminal:

npx react-native init MyApp --template react-native-template-tfjs

This command may require a little explaining:

  • The npx command (which comes with Node.js/npm) downloads the react-native command from the npm package registry and runs it.
  • The init sub-command is used to initialise a new project with name MyApp (change this as you wish).
  • Finally the --template option specifies the template to use. In this case, we want to use react-native-template-tfjs which I recently created. It will find this via the npm package registry and use the template to create the new project.

Your project will be created in MyApp directory. In the MyApp directory you can then run the app with react-native run-ios or react-native run-android, so long as you’ve already configured your environment for these platforms.

Other than having TensorFlow.js configured, your new project is the same as the standard React Native starter project.

Using TensorFlow.js in React Native

You may notice, when the app first loads, it displays “TensorFlow.js v1.5.2 is loading” (for a very short time).

This is because React Native WebGL backend is not immediately available for use after import. We can use the tf.ready() method to get a promise that will return when the backend is ready to use. The template uses this method to update a state variable isTfReady. It does this using React useState and useEffect hooks:

const [isTfReady, setTfReady] = useState(false);

useEffect(() => {
  async function waitForTensorFlowJs() {
  await tf.ready();
    setTfReady(true);
  }
  waitForTensorFlowJs();
}, []);

Don’t worry, this code is included in the template, you don’t have to add it!

When the isTfReady state updates, React Native will trigger a re-rendering of the App component and the message is re-rendered as: “TensorFlow.js v1.5.2 is ready and using backend: rn-webgl”.

React Native app with TensorFlow.js WebGL backend
Our new React Native project, generated with react-native-template-tfjs

The isTfReady state variable is also available for your use: wherever you want to use TensorFlow.js, you can check that it is ready to be used.

Adding functionality to a React Native TensorFlow.js project

Let’s walk through a simple (and admittedly not very exciting) example of reading a value from a simple Tensor calculation. The example we will use is something you can do much more easily with plain JavaScript, but it serves to demonstrate a general pattern we can use when writing TensorFlow.js code in a React Native app.

The plain JavaScript equivalent would be:

const result = 4125 * 1732;

Since this will be a quick calculation, we could include it in our render method and then render it immediately in our JSX code:

<Text>{result}</Text>

However, when working with TensorFlow.js we often need to work with promises, which adds some complexity to our code. React Native rendering cannot work directly with promises, so instead we need to update a state variable once a promise resolves. When it resolves, we update the state, and this triggers a re-render.

First we define a state variable:

const [result, setResult] = useState(null);

The result variable holds our state value, and setResult allows us to set the state of result.

Next we run the calculation with a useEffect hook (which will run after rendering):

useEffect(() => {
  if (isTfReady) {
    const runMultiplication = async (x, y) => {
      const result = await tf.scalar(x).mul(tf.scalar(y)).toArray();
      setResult(result[0]);
    }
    runMultiplication(4125, 1732);
  }
}, [isTfReady]);

We can then render the result, which will only display once the result variable is set:

{typeof result !== ‘undefined’ && (
  <Text>{result}</Text>
)}

The JSX code is slightly more complex than the plain JavaScript version, because we need to guard against the null value prior to the result being ready.

Next steps with TensorFlow.js for React Native – over to you!

If you’re looking to integrate machine learning into a mobile app, there’s never been a better time to start!

In this blog post, I covered project configuration and a simple code example. To build something more significant you’ll need a good understanding of both TensorFlow.js and React Native, plus a good dose of creativity!

If you’ve yet to learn TensorFlow.js, you may like to check out my course Machine Learning in JavaScript with TensorFlow.js. The machine learning techniques in this course can also be used in the React Native environment, although the UI code will need to be adapted because we cannot use HTML and CSS in React Native.

Have fun with machine learning in React Native and let us know what you’re building 🙂

This post © 2020 Justin Emery. Made available under a Creative Commons Attribution-ShareAlike 4.0 Licence.

Justin Emery is the instructor for Machine Learning in JavaScript with TensorFlow.js. He is also a working software developer with a decade of professional experience. He is a multilingual programmer having worked with Java, PHP, and Perl in the past, but now focuses mainly on full-stack JavaScript development. Based in the UK, he is available for software development projects on a freelance basis.

Leave a Comment