Why Machine Learning in Browser?
The browser is the unique platform where the things you build, you can share with anyone with just a link.Those people that open your app don’t have to install any drivers or any software.
The browser is highly interactive and so the user is going to be engaged with whatever you’re building.
Another big thing is that browsers have access to sensors data.All of these sensors are standardize API’s that work on all browsers.
The data that comes from these sensors don’t ever have to leave the client you don’t have to upload anything to the server which preserves privacy.
deeplearn.js and took existing models from python and build interactive things with it.
One example is the style transfer.
Another is ported the character RNN and then build a novel interface that allows you to explore all the different possible endings of a sentence all generated by the model in real time.
Another example is a phone generative model allowed users to explore the interesting dimensions in the embedding space and you can see how they relate to boldness and slanted the font.
There was an educational examples teachable machine that builds this fun little game. That taught people how computer vision models work.So people could interact directly with the webcam.
All above examples use deeplearn.js.
deeplearn.js has become TensorFlow.js
One use case is you can write models directly in the browser and this has huge educational implications think of the playground.
The second use case is a major, you can take a pre-existing model(pre-trained model in python) and you can import it into the browser to do inference.
The third related use case is the same model that you take to do inference you can retrain it potentially with private data that comes from those sensors of the browser in the browser itself.
The browser utilizes WebGL to do fast linear algebra, on top of it.
tensorflow.js has two sets of API’s the Ops API(eager) which used to be
deeplearn.js.It is powered by an automatic differentiation library that is built analogously to eager mode.On top of that, it has high-level API that allows you to use best practices and high-level building blocks to write models.
Google release tools that can take an existing TensorFlow save model or keras model and port it automatically for execution in the browser.