Transfer Learning with TensorFlow Hub Module

Growth on the machine learning side is lagging behind the software engineering side by 15-20 years.This creates a really interesting opportunity.We can look at some of the things that happened at software engineering and think about what kind of impact they may have on machine learning. Looking at software engineering there’s something that’s so fundamental and this idea of sharing code.

Shared Machine Learning


sharing code
Shared code repositories make us immediately more productive.We can search for code, download it, use it, it has these really powerful effects.This actually changes the way we write code.We refactor our code, put it in libraries, share those libraries and this really makes people even more productive.

sharing machine learning code
It’s the same dynamic that TensorFlow creates for machine learning with TensorFlow Hub.With TensorFlow Hub, you can build, share and use pieces of machine learning.

Why shared Machine Learning


Elements of Machine Learning
Anyone who’s done machine learning from scratch knows you need a lot to do, you need an algorithm, data, computer power and expertise and if you’re missing any of these you’re out of luck.

TensorFlow Hub lets you distill all these things down into a reusable package.We call a module.Those modules go into TensorFlow Hub where people can discover them and then easily use them.

TensorFlow Hub
Notice that it’s module instead of a model.A model is big for sharing.If you have a model you can use that model if you have the exact inputs it wants and you expect the exact outputs it provides.If there are any little differences you’re kind of out of luck. Modules are a smaller piece.If you think of a model like a binary and think of a module like a library.

Modules Contain


Module ContainModules contain pre-trained weights and graphs.The module is actually a saved model.It lets us package up the algorithm in the form of a graph.This is package up the weights. You can do things like initializing use assets.Modules are composable, reusable,re-trainable.

TensorFlow libraries make it(Models) very easy to instantiate in your TensorFlow code.So you can compose these in interesting ways.This makes things very reusable.You can produce one of these and share it. These are also retrainable.Once you patch it into your program you can backpropagate through it just like normal.This is really powerful because if you have enough data you can customize the TensorFlow hub module for your own application.

Image Retraining


Let’s look at a specific example of using a TensorFlow Hub for image retraining.Let’s say that we’re gonna make an app that can classify cat from the image. The problem is we have only a couple hundred examples, probably not enough to train an entire image classification model from scratch, but what we could do is start from an existing general-purpose image classification model.

Image Retraining

Most of the high-performance model is trained on millions of examples.They can easily classify thousands of categories.We want to reuse the architecture and the trained weights of that model without the classification layers and in that way we can add our own cat classifier on top.We can train it on our cat examples and keep the reused weights fixed.

Image Modules

You can find a list of all of the newly released the state-of-the-art, well-known image modules. some of them include the classification layers and some of them remove them just providing a feature vector as output.We’ll choose one of the feature vectors, for this NASNet which is a state-of-the-art image module that was created by a neural architecture.

So you just paste the URL of a module and TensorFlow Hub takes care of downloading the graph and all of its weights and importing it into your model in one line.You’re ready to use the module like any function.

Here, We just provide a batch of inputs and we get back our feature vectors.We add a classification layer on top and we output our predictions.But in that one line, you get a huge amount of value in this particular case more than 62,000 hours of GPU time.

Available Image Modules

NASNet is available in a large size as well as a mobile size module and there’s also the new progressive PNASNet .A number of new MobileNet modules for doing on-device image classification as well as some industry standard ones like Inception, and ResNet that complete list is available here.All those modules are pre-trained using the TF-slim checkpoints and already used for classification or as feature vector inputs to your own model.

Text Classification


We’d like to know whether a movie review is a positive or negative sentiment.One of the great things about TensorFlow Hub is that it packages graph with the date.

Restaurant ReviewText and embedding modules contain all of the pre-processing and included a thing like normalizing and tokenizing operations.We can use a pre-trained sentence embedding module to map a full sentence to an embedding vector.So If we want to classify some movie reviews then we just take one of those sentence embedding modules.we add our own classification layer on top and then we train with our review.We keep the sentence modules weights fixed.

TensorFlow Hub has a number of different text modules.It has neural network language models that are trained on Google news for English, Japanese, German, and Spanish. It has  word2Vec model trained on Wikipedia as well a new module called ELMO that models the characteristics of word use.

Universal Sentence Encoder

Universal sentence encoder it’s a sentence level embedding module.It is trained on a variety of tasks and it enables a variety of tasks in other words universal. So some of the things that are good for our semantic similarity doing custom text classification clustering and semantic search but the best part about is It requires little training data.

We just past that URL and TensorFlow Hub downloads the module and inserts it into your graph.This time we’re using the text embedding column that way we can feed it into an estimator to do the classification part.Like with the image retraining example this module can be fine-tuned with your model by setting trainable to true.You have to lower the learning rate.So that you don’t ruin any existing weights that are in there but it’s something that’s worth exploring if you have enough data.

Module URL


Module is a program So make sure what you’re executing is from a location that you trust. In this case, the module is from tfhub.dev.It provided modules like NASNet and a universal sentence encoder. TensorFlow create a place where you can publish the modules that you create.In this case, Google is the publisher and universal sentence encoder is the name of the module and the version number is 1.So Tensorflow Hub considers modules to be immutable that way you don’t have to worry about the weight changing between training sessions.So All of the module URLs on tfhub.dev include a version number and you can take that URL and past it into the browser and see that complete documentation for any module that’s hosted on tfhub.dve.

Conclusion

TensorFlow also has modules for other domains besides text classification and image retraining like generative image module. TensorFlow Hub added another module that was based on deep local features network which can identify the key points of landmark images both of those have great colab notebooks.

 

References

TensorFlow Hub (TensorFlow Dev Summit 2018)

 

Leave a Reply

Your email address will not be published. Required fields are marked *