Month: July 2017

Google Cloud Speech API in Android APP

brijeshthumar / / Machine Learning APIs

So many of you have used the “Ok Google” functionality on your phone.So what the Speech API does is it lets developers integrate that functionality into their own applications, lets you do speech to text transcription in over 80 languages.

There’s a lot of potential use case where you may want to combine different cloud machine learning APIs.In this case, they’re using the Speech API.So whenever the two people don’t speak the same language, They’ll transcribe the audio with the Speech API and then they translate it into another person’s host language with the Translation API.

But it also works in streaming mode.So if you want to send it a continuous stream of audio, you can send it to a stream of audio and it returns transcriptions as that audio is coming in.once you’re done transcribing your text, you may want to do more analysis on your text.In this tutorial, we are going to learn Streaming Recognition to perform speech recognition.

So the best way to see how the speech API works is through a demo.

Prerequisites

  • Google Cloud Platform account(You can use 12 months free trial)

1.Acquiring an API Key

To Use Google Cloud Speech API services in the app, you need a Service account keys. You can get one by creating a new project in the Google Cloud Platform console.

Once the project has been created, go to API Manager > Dashboard and press the Enable API button.

Enable Google Cloud Speech API

2.Creating a New Android Project

Google provides client libraries in a number of programming languages to simplify the process of building and sending requests and receiving and parsing responses.

Add the following compile dependencies to the app build.gradle:

Set Up to Authenticate With Your Project’s Credentials

This Android app uses JSON credential file locally stored in the resources.we just put the Service Account in the client for ease of use. The app still gets an access token using the service account credential, and use the token to call the API.

In order to get Service Account Key, visit the Cloud Console, and navigate to: API Manager > Credentials > Create credentials > Service account key > New service account. Create a new service account, and download the JSON credentials file. Put the file in the app resources as app/src/main/res/raw/credential.json.

Generate Google Cloud Service Account

Streaming Speech API Recognition Requests

A streaming Speech API recognition call is designed for real-time capture and recognition of audio, within a bi-directional stream. Your application can send audio on the request stream and receive interim and final recognition results on the response stream in real time. Interim results represent the current recognition result for a section of audio, while the final recognition result represents the last, best guess for that section of audio.

Streaming requests

You send both the configuration and audio within a single request, calling the streaming Speech API requires sending multiple requests. The first StreamingRecognizeRequest must contain a configuration of type StreamingRecognitionConfig without any accompanying audio. Subsequent StreamingRecognizeRequests sent over the same stream will then consist of consecutive frames of raw audio bytes.

A StreamingRecognitionConfig consists of the following fields:

  • config – (required) contains configuration information for the audio, of type RecognitionConfig and is the same as that shown within synchronous and asynchronous requests.
  • single_utterance – (optional, defaults to false) indicates whether this request should automatically end after speech is no longer detected. If set, the Speech API will detect pauses, silence, or non-speech audio to determine when to end recognition. If not set, the stream will continue to listen and process audio until either the stream is closed directly, or the stream’s limit length has been exceeded. Setting single_utterance to true is useful for processing voice commands.
  • interim_results – (optional, defaults to false) indicates that this stream request should return temporary results that may be refined at a later time (after processing more audio). Interim results will be noted within responses through the setting of is_final to false.

Streaming responses

Streaming speech recognition results are returned within a series of responses of type StreamingRecognitionResponse. Response consists of the following fields:

  • speechEventType contains events of type SpeechEventType. The value of these events will indicate when a single utterance has been determined to have been completed. The speech events serve as markers with in your stream’s response.
  • results contains the list of results, which may be either interim or final result, of type
    StreamingRecognitionResult. The results list contains following the sub-fields:

    • alternatives contains a list of alternative transcriptions.
    • isFinal indicates whether the results obtained within this list entry are interim or are final.
    • stability indicates the volatility of results obtained so far, with 0.0 indicating complete instability while 1.0 indicates complete stability. Note that unlike confidence, which estimates whether a transcription is correct, stability estimates whether the given partial result may change. If isFinal is set to true, stability will not be set.

 

Download this project from GitHub

Related Post

Google Cloud Natural Language API in Android APP

Google Cloud Vision API in Android APP

Speech Recognition Using TensorFlow

 

 

Google Cloud Natural Language API in Android APP

brijeshthumar / / Machine Learning APIs

When you want to do something more with your text once you’ve transcribed it.You might want to analyze it.That’s where the Natural Language API comes into play.The Natural Language API lets you extract entities, sentiment, and syntax from your text.Real world example is customer feedback platform.They enable all of their users to do is gather feedback from their app’s user as they’re going through their application.So Customer their make sense of all this open-ended feedback.It’s that open-ended text feedback that’s much more difficult for them to make sense of.That’s where the Natural Language API comes into play.Entity and Syntax annotation to pull out the key subjects and terms from the feedback, and then if necessary, route those to the right person in real time to respond to the feedback.

In this tutorial, I’ll introduce you to the Cloud Natural Language platform and show you how to use it to analyze text.

Prerequisites

  • Google Cloud Platform account(You can use 12 months free trial)

 

1.Acquiring an API Key

To Use Google Cloud Natural Language API services in the app, you need an API key. You can get one by creating a new project in the Google Cloud Platform console.

Once the project has been created, go to API Manager > Dashboard and press the Enable API button.

Enable Natural Language API

To get API key, go to the Credentials tab, press the Create Credentials button, and select API key.

Google Cloud Vision API Key

2.Creating a New Android Project

Google provides client libraries in a number of programming languages to simplify the process of building and sending requests and receiving and parsing responses.

Add the following compile dependencies to the app build.gradle:

Add INTERNET permission in the AndroidManifest.xml file.

 

To interact with the API using the Google API Client library, you must create a CloudNaturalLanguage object using the CloudNaturalLanguage.Builder class. Its constructor also expects an HTTP transport and a JSON factory.

Furthermore, by assigning a CloudNaturalLanguageRequestInitializer instance to it, you can force it to include your API key in all its
requests.

All the text you want to analyze using the API must be placed inside a Document object. The Document object must also contain configuration information, such as the language of the text and whether it is formatted as plain text or HTML. Add the following code:

Next, you must create a Features object specifying the features you are interested in analyzing. The following code shows you how to create a Features object that says you want to extract entities and run sentiment analysis only.

Use the Document and Features objects to compose an AnnotateTextRequest object, which can be passed to the annotateText() method to generate an AnnotateTextResponse object.

 

Entity Analysis

So I’ve got the sentence and I sent it to the entity extraction endpoint of the Natural Language API And it return all of these as entities in my text. So we can see that each entity, we get the name of the entity, in this case, Google.The type of the entity is organization.Then we get back some metadata.MID ID that maps to Google’s Knowledge Graph.If you want to get more information about the entity, you can call Google’s Knowledge Graph API, passing it this ID.We also get the Wikipedia URL for this particular entity.

You can extract a list of entities from the AnnotateTextResponse object by calling its getEntities() method.

Analyze Entity

 

Sentiment Analysis

Analyze the sentiment of your text.If we have this restaurant review,

The food at that restaurant has stale,I will not be going back.

If I worked at this restaurant, I ’d and potentially follow up with this customer to see why they didn’t like it.But it’s likely that I would have lots and lots of reviews, and I probably wouldn’t want to read each one manually.I might want to flag the most positive and most negative once and then respond just to those. So we get two number back from the Natural Language API to help us do this.The first thing we get back is score, which will tell us on a scale from -1 to 1 how positive or negative is this text? In this example, we get negative 0.8, which is almost fully negative.Then we get magnitude, which tells us regardless of being positive or negative, how strong is the sentiment in this text?And this is a range from 0 to infinity, and it’s normalize based on the length of the text.So we get a pretty small number here,0.8 because this is just a small piece of text.

You can extract the overall sentiment of the transcript by calling the getDocumentSentiment() method. To get the actual score of the sentiment, however, you must also call the getScore() method, which returns a float.

analyze sentiment

 

Download this project from GitHub

Related Post

Google Cloud Vision API in Android APP

Google Cloud Speech API in Android APP

Google Cloud Vision API in Android APP

brijeshthumar / / Machine Learning APIs

Smartphone users are starting to look for smarter apps. As an Android Developer, you probably lack the resources needed to create automation, artificial intelligence, and machine learning apps from scratch.

Google recently launched a Cloud Machine Learning platform, which offers Neural Networks that have been pre-trained model to perform a variety of tasks.You can use them by simply making a few REST API calls or Client library.

In this tutorial, I’ll introduce you to the Cloud Machine Learning platform and show you how to use it to create a smart Android app that can recognize the real-world object.

Prerequisites

  • Device running Android 4.4+
  • Google Cloud Platform account(You can use 12 months free trial)

1.Acquiring an API Key

To Use Google Vision API services in the app, you need an API key. You can get one by creating a new project in the Google Cloud Platform console.

Once the project has been created, go to API Manager > Dashboard and press the Enable API button.

Enable Google Vision API

To get API key, go to the Credentials tab, press the Create Credentials button, and select API key.

Google Cloud Vision API

2.Creating a New Android Project

Google Cloud Vision API use to understand the content of an image by machine learning models using  REST API. It quickly classifies images into thousands of categories, detects objects and faces within images, and finds and reads printed words contained within images.

Google provides client libraries in a number of programming languages to simplify the process of building and sending requests and receiving and parsing responses.

Add the following compile dependencies to app build.gradle:

Add INTERNET permission in the AndroidManifest.xml file.

Step 1: Create an Intent

Creating a new intent with the ACTION_IMAGE_CAPTURE action and passing it to the startActivityForResult() method, you can ask the default camera app of the user’s device to take pictures and pass them on to your app. Add the following code to your Activity class:

Receive the images captured by the default camera app in onActivityResult() method of activity class. you’ll have access to a Bundle object containing all the image data. You can render the image data by simply converting it into a Bitmap and passing it to the ImageView widget.

Step 3: Encode the Image

The Vision API cannot use Bitmap objects directly. It expects a Base64-encoded string of compressed image data.To compress the image data, you can use the compress() method of the Bitmap class. As its arguments, the method expects the compression format to use, the output quality desired, and a ByteArrayOutputStream object. The following code compresses the bitmap using the JPEG format.

Step 4: Create Feature

The Feature indicates what type of image detection task to perform. Describe the type of Vision tasks to perform over images by using Features. Features encode the Vision vertical to operate on and the number of top-scoring results to return.

This is the Java data model class that specifies how to parse/serialize into the JSON that is transmitted over HTTP when working with the Cloud Vision API.

Step 5: Create Request

Create the request for performing Vision tasks over a user-provided image, with user-requested features.

Step 6: Process the Image

Now, you need to interact with the Vision API. Start by creating a HttpTransport and VisionRequestInitializer that contains your API key:

CLOUD VISION API FEATURES

Derive insight from images with Google powerful Cloud Vision API

1.Label Detection

The Vision API can detect and extract information about entities within an image.Labels can identify objects, locations, activities, animal species, products, and more.

Vision API Label Detection

2.Landmark Detection

Landmark requests detect well-known natural and human-made landmarks and return identifying information such as an entity ID (that may be available in the Google Knowledge Graph), the landmark’s name and location, and the bounding box that surrounds the landmark in the image.

Vision API Landmark Detection

3.Logo Detection

Logo detection requests detect popular product and corporate logos within an image.

4.Safe Search Detection SAFE_SEARCH_DETECTION

Safe Search requests examine an image for potentially unsafe or undesirable content. Likelihood of such imagery is returned in 4 categories:

  • adult indicates content generally suited for 18 years plus, such as nudity, sexual activity, and pornography (including cartoons or anime).
  • spoof indicates content that has been modified from the original to make it funny or offensive.
  • medical indicates content such as surgeries or MRIs.
  • violent indicates violent content, including but not limited to the presence of blood, war images, weapons, injuries, or car crashes.Vision API Safe Search

5.Image Properties

An image properties request returns the dominant colors in the image as RGB values and percent of the total pixel count.

Vision API Image Property

 

Conclusion

In this tutorial, you learned how to use the Cloud Vision, which is part of the Google Cloud Machine Learning platform, in an Android app. There are many more such APIs offered by the platform. You can learn more about them by referring to the official documentation.

 


Download this project from GitHub

 

Related Post

Android TensorFlow Machine Learning

Google Cloud Natural Language API in Android APP

Google Cloud Speech API in Android APP

Phone Number Authentication with Firebase Auth

brijeshthumar / / Firebase

Growing Globally with Phone Number identity

Identity is a really important part of growth.If people can’t or won’t login to your app, then you don’t get a user.You don’t have growth.

Why phone numbers?

Using phone number as an effective identity for growth.Sign-in is such an important part of your growth funnel.Over 90% of people who have issue login into an app is going to leave.

1.They’re global.

Every single person in the world who has a smartphone has a phone number that they have memorized.

2 Actually, in a lot of emerging markets, We’ve learned that people prefer to use their phone as their identity instead of an email address or another form of identification.

3 Phone numbers are higher quality.They give you higher quality users.Because they’re more expensive to mass produce, they’re naturally resistant to the kinds of abuse and fraud you might see more often in another type of identification.Most importantly very low-friction sign-in for apps.

Why is login so hard to use?

We know as developers that passwords don’t increase security as much as they frustration and friction.

As a developer, we know that login is harder than it needs to be dealing with login is hard.There are UX implications.There are security implications.

Firebase Phone Auth

Today’s apps need to be able to support high-quality,low-friction authentication and engage with them in new ways.That makes phone number auth really valuable.

A lot of the apps that actually don’t have phone number authentication today, bBuilding it in the first place can be a real struggle.The first thing you’ll need to do is integrate with an SMS provider, and get coverage with carriers to send SMS.Then,you’re going to want to expand your offering globally to support all the different areas that your user might come from, making sure you’re perfectly localized along the way.

After all of that, you need to build secure authentication.That means generating a one time code, verifying it from a user, minting cryptographically signed tokens, persisting session, writing your anti-abuse logic. Firebase launched its phone authentication APIs.

The first thing that Firebase allow is it enables basic sign-in and sign-up flows. On Android Firebase actually, allow a few and built in enhancements.In the previous post you understand about the SMS retriever API, which actually increases overall conversion from users, SO they don’t have to end up with the SMS app and parsing it.This is going to come built into firebase Android APIs.Finally, Firebase has a really new cool piece of functionality called instant verification on Android.

Firebase enables basic sign-in and sign-up flows.

The first thing that Firebase want to make sure enable your app to do is allow you to sign in and sign up your users.All a user would need to do is enter their phone number, and you can go ahead and pass it over to Firebase.

Firebase will validate the phone number, handle any normalization logic that needs to happen, if this is a new user, or a returning one, and go ahead and generate a one-time code associated with this request.

The new user will simply pass back the code into your application, and which you would just pass right back on to Firebase, and Firebase will verify it.Firebase will handle any of the complex logic, and deduplication, if you send multiple SMS by mistake or any of the other edge cases that might arise.The user will go ahead and enter their code.Firebase will validate it.Firebase will go ahead and mint cryptographically signed tokens, persist sessions the whole authentication.You won’t have to worry about how any of it.

Auto-retrieval

Auto-retrieval works very similarly to we discussed in the previous post.It’s comes built in the user phone auth APIs.For auto retrieval, if the user goes ahead and click sign with the phone number.This time they can actually select their phone number if you integrate with hint selector.It’ll be auto-populated into the application.Firebase will validate it once more. Firebase will generate the code.But this time, when firebase actually sends the code in the SMS, Firebase will be able to pick it up from the background.Firebase auto-retrieve it directly, parse it out and populate it into the application.The best part of all of this is your user never actually had to leave your application, go to an SMS tab, wait for it and then copy and paste the code into your UI.They stayed in your application the entire time.It’s a really seamless experience for them overall.That’s all that you ever to do.All this powerful functionality is also really easy to use.

Configuration

As a pre-requisite, ensure your application is configured for use with Firebase.

Then, add the FirebaseUI auth library dependency.

The second thing you need to do is just enable phone auth within your project in the Firebase console.

 

Enable Firebase Phone Auth

Now Firebase headless APIs don’t enforce any UI, so they give you programmatic access to our underlying infrastructure that you can use in any way that you choose.It’s really easy to use.On Android, in order to get all the awesome functionality the actual ability to generate and send codes, the auto-retrieval, even instant verification, comes down to just one single API call, and the implementation of the callbacks that you care about.

What’s about UI?

Building authentication UI can be pretty complex.You need to handle all sorts of edge cases across all the different platforms your application might support.You want to make sure that your login UI is actually high performance.That will give you good conversion rates.FirebaseUI wrote out all the code for different phone authentication flows that.

 

The first thing you’ll notice is Firebase UI integrates with hint selector directly.So I didn’t write any additional code for this.It’s able to pick up user’s phone number from the device, and I can just simply enter it directly into the app.

Now when you hit verify phone number, it’ll go ahead and send a code which will directly get right off the SMS.The code was sent to your device immediately written from the SIM.All I did was really tap my phone number and everything else was taken for me.The SMS was delivered.It was passed.It was taken.I never left the app.

Instant Verification

Instant verification works exactly like it sounds like.It allows you to verify the phone number without sending an SMS to the user, it appears instantly.Let’s How it works?

1.User Select Number

This time, When they enter their phone number, and you call APIs, Firebase is going to do the first check.That check will go ahead and see if firebase has verified this phone number on the device recently.If it has, it can actually instantly verify the phone number the next time around without needing to send any additional SMS.It thinks that this means there is no wait time, on SMS, or anything else, just pure simplicity verify.

 

Imagine that user coming back to the application, maybe after a week or two.This time user going to instant verification.When the user goes to the application, let’s sign in with a phone number again.It’ll pull up the phone number for user device just like last time.The user just going to do single tap log-in.Where I go ahead and select a phone number, It sees that I’d already been verified in the past and that It’s.No SMS was sent, nothing else.But instead, Firebase was able to just validate that we verified it, and logged them in directly into te application.

 

Related Post

SMS Retriever API

SMS Retriever API

brijeshthumar / / Firebase

Automatic SMS Verification with the Phone Selector and SMS Retriever API

Enable to support phone numbers for authentication, if you have an existing backend infrastructure that you are already using for verifying the phone numbers.

Using SMS Retriever API, you can perform SMS-based verification in your app automatically, without requiring the user to manually type verification codes, and without requiring any extra app permissions.

Most apps perform verification by sending an SMS to the user’s device with a one-time verification code that the application that verifies.So the app has to get the phone number of the device and then read the verification code from the SMS.

The Google Play Services SDK(10.2 and newer) offering you to enable read the phone number and the verification SMS automatically without requiring these extra permissions.

For reading the phone number, Android has the Phone Selector API.For verifying the phone number, Android has the SMS Retriever API.These APIs are part of the Google Play Service SDK.

1.Getting the phone number of the device

Don’t ask the user to enter these values.Manually inputting the phone number is painful.It adds a lot of friction for the user to provide a phone number to your app.It’s also error-prone.

Phone Selector API

Phone Selector API provides the Phone number to your app with a much better user experience and no extra permissions.Using this API, you can launch a dialogue, which shows the phone numbers on the device to the user.

Phone Number HintSelector

First, you create a hint request object and set the phone number identifier supported field to true.

Then, you get a pending intent from that hint request for the phone number selector dialogue.

Once the user selects the phone number, that phone number is returned to your app in the onActivityResult().

2.Reading the verification code automatically using SMS retriever API

SMS Retriever API provides you the message content to your app without requiring any extra permissions.The key part is that it provides you only the message targeted your app.You have to verification code in your SMS message and include app-specific hash.This app-specific hash is a static hash that you can just include in the SMS template without requiring many code changes.

SMS Retriever API

Start the SMS retriever

This makes it wait for one matching SMS, which includes the app-specific hash.

Once the SMS with the app-specific hash is received on the device, it is provided to your app via broadcast.

In your broadcast receiver, you can get the message content from the extras.Once you have the message content, you can extract the verification code, and verify the code just like you would normally do.

Register this BroadcastReceiver with the intent filter.

After starting the SMS retriever, you can just send the SMS with the verification code and the app-specific hash to the phone using any backend infrastructure of yours.

3.Construct a verification message

Construct the verification message that you will send to the user’s device. This message must:

  • Be no longer than 140 bytes
  • Begin with one of the following strings:
    • [#]
    • Two consecutive zero-width space characters (U+200B)
  • End with an 11-character hash string that identifies your app

Otherwise, the contents of the verification message can be whatever you choose. It is helpful to create a message from which you can easily extract the one-time code later on. For example, a valid verification message might look like the following:

Computing your app’s hash string

ou can get your app’s hash string with the AppSignatureHelper class from the SMS retriever sample app. However, if you use the helper class, be sure to remove it from your app after you get the hash string. Do not use hash strings dynamically computed on the client in your verification messages.

 

Related Post

Phone Number Authentication with Firebase Auth

TensorFlow

brijeshthumar / / TensorFlow

TensorFlow is an open source software library for Machine Learning.It was originally developed by researchers and engineers working on the Google Brain team within Google’s Machine Intelligence Research organization for the purposes of conducting machine learning and deep neural networks research.

TensorFlow is programming paradigm and some of its main features are that numeric computation is expressed as a computational graph backbone.TensorFlow program is going to be a graph where the graph nodes are the group to be operations, shorthand as an operation in your code.They have any number of inputs and a single output.The edges between our nodes are going to be tensors that flow between them.

The best way of thinking about what tensors are in practice is as n-dimensional arrays.The advantage of using flow graphs as the backbone of your deep learning framework is that it allows you to build complex models in terms of small and simple operations.This is going to make your gradient calculation extremely simple when we get to that.You’re going to be very grateful for the automatic different when you’re coding large models in your project and in the feature.

Another way of thinking about a TensorFlow graph is that each operation is a function that can be evaluated at that point.

Neural Network computational graph

 

Neural network with one hidden layer what it’s computational graph in TensorFlow might look like.

 

h=ReLU(Wx+b)

So we have some hidden layer that we are trying to compute, as the ReLU activation of some parameter matrix W times some input x plus a bias term.

ReLu is an activation function standing for rectified linear unit.We are applying some nonlinear function over our linear input that is what gives the neural networks their expressive function.The ReLU takes the max of your input and zero.

We have variables b and W.We have a placeholder with the x. Nodes for each of the operations in our graph.

Variables are going to be stateful nodes which output their current value.In our case, Variable b and W is retained their current value over multiple executions.It’s easy to restore saved values to variables.

Variable has a number of other useful features.they can be saved to your disk during and after training. That allows people from different companies and group to save, store, and send over their model parameter to other people.They also make gradient update by default.It will apply over all of the variables and your graph.

The variable is the things that you wanna tune to minimize the loss.It is really important to remember that variable in the graph like b and W are still operations.

All of your node in the graph are operations.When you evaluate the operation that is these variables in our run time you will get the value of those variables.

Placeholders(x) is nodes whose value is fed in at execution time. Placeholder values that we’re going to add into our computation during training.So this going to be our input.

So for placeholders, we don’t give any initial values.we just assign a data type, and we assign a shape of tensor so the graph still knows what to computer even though it doesn’t have any stored values yet.

The third type of node is mathematical operations. This going to be your matrix multiplication, your addition, and your ReLU. All of these are nodes in your TensorFlow graphs.It’s very important that we’re actually calling on TensorFlow mathematical operations as opposed to Numpy operations.

Related Post

TenserFlow Lite

Install TensorFlow

Train Image classifier with TensorFlow

Train your Object Detection model locally with TensorFlow

Android TensorFlow Machine Learning

 

TenserFlow Lite

brijeshthumar / / Uncategorized

It’s a new API for neural network processings inside Android and that will be added to the Android framework.The purpose of adding a new API is to encapsulate and have an abstraction layer for the hardware accelerators such as GPU, DSP, and ISP.Modern smartphones have such a powerful computing resource other than CPU, such as GPU or DSP.Especially the DSP is designed to do a massive amount of the matrix and vector calculations, So it’s much faster to use DSP or GPUs to do the neural networks inference on it, rather than using a CPU. But right now, If you want to do that, you have to go directly into the library provided by the hardware vendors and building some binaries by yourself.It’s a tedious task.So instead, Android will be providing a standard API so that developers don’t have to be aware of any of the hardware accelerators from each individual vendor.On top of the Neural Network API, Android will be providing TensorFlow Lite.That will be a new TensorFLow runtime optimized for mobile and embedded applications.

TensorFlow Lite is designed for smart and compact mobile or embedded applications. It is designed to be combined with the Android Neural Network API.So all you have to do is to build your model with TensorFlow Lite, and that’s it. Eventually, you will be getting all the benefits you could get from the Android Neural Network API, such as the hardware acceleration.

TensorFlow Lite and the neural network API will be made available later in an update to O this year.

Related Post

Introduction TensorFlow Machine Learning Library

Install TenserFlow

Train Image classifier with TensorFlow

Train your Object Detection model locally with TensorFlow

Android TensorFlow Machine Learning

 

Android TensorFlow Machine Learning

brijeshthumar / / TensorFlow

TensorFlow is an open source software library for machine learning, developed by Google and currently used in many of their projects.

In this article, we will create an Android app that can recognize five types of fruits.

Machine Learning

There have been so many buzzwords such as AI, Machine Learning, Neural Network, or Deep Learning.What’s the difference?

AI or Artificial Intelligence–you can say that is a science for making smart things like building an autonomous driving car or having a computer drawing a beautiful picture or composing music.One way to realize that vision of AI is in Machine Learning.Machine learning is a technology where you can a computer train itself, rather than having human programmers instructing every step, to process the data by itself.

One of the many different algorithms in MI is a neural network.Since around 2012, Google has been seeing a big breakthrough in the world of the neural network, especially for image recognition, voice recognition, or natural language processing and many other applications.

Neural Network

You can think of it just like a function in mathematics or a function in the programming language.So you can put any kind of data as an input and do some matrix operation or calculations inside neural networks. You would get an output vector which has the many labels or speculative values.

Neural Network

For example, if you have a bunch of images, you can train the neural network to classify which one is the image of a cat or the image of a dog, this is just one example of the use cases of neural networks.You can apply the technology to solve any kind of business problems you have.
There are so many possible use cases for the combination between ML and mobile applications, starting from image recognition, OCR, speech-to-text, and text-to-speech, translation.You can apply machine learning to mobile-specification applications such as motion detection or GPS location tracking.

Why do you want to run machine learning model inside your mobile applications?

By using the machine learnings, you can reduce the significant amount of traffic, and you can get much faster responses from your cloud services.Because you can extract the meaning from the raw data.For example, if you are using machine learning for image recognition, the easiest way to implement that is to send all the raw image data taken by the camera to the server. But instead, you can have the machine learning model running inside your mobile application so that your mobile application can recognize what kind of object is in each image.So that you can just send the label, such as a flower or human face, to the server.That can reduce the traffic to 1/10 or 1/100 It’s a significant amount of saving of traffic.

Build an application that is powered by machine learning

The starting point could be the TensorFlow, which is the open-source library for machine intelligence from Google.TensorFlow is the latest framework for building machine learning or AI-based service developed in Google.Google open source it in November 2015. TensorFlow is the most popular framework for building neural networks or deep learning in the world.One benefit you could get with TensorFlow is easy of development.So It’s really easy to get started.You can just write a few lines of Python code.

TensorFlow is very valuable for people like me because I don’t have any sophisticated mathematical background.So when you started reading the textbook on neural networks, you found many mathematical equations on the textbook, like differentiation backpropagation and gradient descent.You really didn’t want to implement everything by yourself.Instead, now you can just download TensorFlow, Where you can write a single line of Python code, like GradientDescentOptimizer.That single line of code can encapsulate all these obfuscated algorithms such as gradient descent, backpropagation, or any other latest algorithm implemented by the Google Engineers.So you yourself don’t have to have the skill set to implement the neural network technologies from scratch.The main benefits of the TensorFlow is the portability and scalability.

Implement TensorFlow in Android

Android just added a JSON integration, which makes step a lot, a lot easier.Just add one line to the build.gradle, and the Gradle take care or the rest of steps.Under the library archive, holding TensorFlow shared object is downloaded from JCenter, linked against the application automatically.

Android release inference library to integrate TensorFlow for Java Application.

Add your model to the project

We need the pre-trained model and label file.In the previous tutorial, we train model.which does the object detection on a given image.You can download the model from here.Unzip this zip file, we will get retrained_labels.txt(label for objects) and rounded_graph.pb (pre-trained model).

Put retrained_labels.txt and rounded_graph.pb into android/assets directory.

 

At first, create TensorFlow inference interface, opening the model file from the asset in the APK.Then, Set up the input feed using Feed API.On mobile, the input feed tends to be retrieved from various sensors like a camera, accelerometer, Then run the interface, finally, you can fetch the results using fetch method over there.You would notice that those calls are all blocking calls.So you’d want to run them in a worker thread, rather than the main thread because API would take a long time.This one is Java API.you can use regular C++ API as well.

Download this project from GitHub

 

Related Past

Google Cloud Vision API in Android APP

Introduction TensorFlow Machine Learning Library

TenserFlow Lite

Train Image classifier with TensorFlow

Train your Object Detection model locally with TensorFlow

Speech Recognition Using TensorFlow

 

 

 

Train Image classifier with TensorFlow

brijeshthumar / / TensorFlow

In this post, we’re going to train an Image Classifier for the Android Machine Learning demo app. You’ll need to install TensorFlow and you’ll need to understand how to use the command line.

What is Machine Learning? and Why important?

You can think of Machine Learning as a subfield of AI.Machine Learning is the study of algorithms that learn from examples and experience instead of relying on hard-coded rules.So That’s the state-of-the-art.
Can you write code to tell the difference between an apple and an orange? Takes file as input, does some analysis, and outputs the type of fruit.

We’re going to train a classifier.For now, you can think of a classifier as a function.It takes some data as input and assigns a label to It as output.

The technique to write the classifier automatically is called supervised learning.It begins with the example of the problems you want to solve.

To use supervised learning, we’ll follow a few standard steps.

1.Collect training data

We’re going to write a function to classify a piece of fruit Image. For starters, it will take an image of the fruit as input and predict whether it’s an apple or oranges as output.The more training data you have, the better a classifier you can create (at least 30 images of each, more is better).

Image classifier folder

We will create a ~/tf_files/fruits folder and place each set of jpeg images in subdirectories (such as ~/tf_files/fruits/apple, ~/tf_files/fruits/orange etc)

A quick way to download multiple images at once is to search something on Google Images, and use a Chrome extension for batch download.

2.Train an Image Classifier with TensorFLow for Poets

You want to build a classifier that can tell the difference between a picture of an Apple and Orange.TensorFlow for Poets This is a great way to get started learning about and working with image classification.
To train our classifier we’ll basically just need to run a couple of scripts.To train an image classifier with TensorFlow for Poets, we’ll only need to provide one thing–training data.

The classifier we’ll be using is called a neural network.At a high level, that’s just another type of classifier, like the nearest neighbor one wrote last lime.The different a neural network can learn more complex functions.

Training Inception

Step1: The retrain.py script is part of the TensorFlow repo.You need to download it manually, to the current directory(~/tf_files):

download retrain.py
Now, we have a trainer, we have data(Image), so let’s train! We will train the Inception v3 network.

Step2: Before starting the training, active  TensorFlow.

active tensorflow environment

Step3: Start your image retraining with one big command.

Train Image classifier

These commands will make TensorFlow download the inception model and retrain it to detect images from ~/tf_files/fruits.

Train Image classifier accuracy

This operation can take several minutes depending on how many images you have and how many training steps you specified.

The script will generate two files: the model in a protobuf file (retrained_graph.pb) and a label list of all the objects it can recognize (retrained_labels.txt).

retrained grape files

Clone the Git repository for test model

The following command will clone the Git repository containing the files required for the test model.

Copy tf file

The repo contains two directories: android/, and scripts/

1.android/: Directory contains nearly all the files necessary to build a simple Android app that classifies images.

2.scripts/: Directory contains the python scripts. These include scripts to prepare, test and evaluate the model.

Now copy the tf_files directory from the first part, into /tensorflow-for-poets-2 working directory.

Test the Model

The scripts/ directory contains a simple command line script, label_image.py, to test the network.

test trained model

Optimize model for Android

TensorFlow installation includes a tool, optimize_for_inference, that removes all nodes that aren’t needed for a given set of input and output nodes.

Optimize for inference

It creates a new file at tf_files/optimized_graph.pb.

Make the model compressible

The retrained model is still 84MB in size at this point. That large download size may be a limiting factor for any app that includes it.

Neural network operation requires a bunch of matrix characterizations, which means tons of multiply and add operations.current mobile devices are capable of doing some of them with specialized hardware.

Quantization

Quantization is one of the techniques to reduce both memory footprint and computer load.Usually, TensorFlow takes a single precision floating value for input and math, and also output as well.As you know, the single precision floating point takes 32 bits each.W can reduce the precision to 16 bits,8 bit, or even less while keeping a good result, just because our learning process involves some noise by nature.Adding some extra noise wouldn’t matter much.So quantized weight is the optimization for storage size, which reduces the precision of the constant node in the graph file.

Quantize Image

Quantized calculations.

With the quantized calculations, we can reduce computing precision by using the quantized value directory.This one is good for first memory bandwidth, which is a limiting factor in mobile devices.Also, hardware can handle these precision values faster than single precision floating values.

Now use the quantize_graph script to apply changes:

quantize graph

It does this without any changes to the structure of the network, it simply quantizes the constants in place.It creates a new file at tf_files/rounded_graph.pb.

 

Every mobile app distribution system compresses the package before distribution. So test how much the graph can be compressed:

 

Quantize compare

You should see a significant improvement. I get 73% optimize model.

 

Related Post

Train your Object Detection model locally with TensorFlow

Android TensorFlow Machine Learning

Introduction TensorFlow Machine Learning Library

TenserFlow Lite

 

Install TensorFlow

brijeshthumar / / TensorFlow

Installing TensorFlow for CPU on Ubuntu using virtualenv.

This blog explains how to install TensorFlow on Ubuntu using Virtualenv. Virtualenv is a virtual Python environment isolated from other Python development, incapable of interfering with or being affected by other Python programs on the same machine. To start working with TensorFlow, you simply need to “activate” the virtual environment. All in all, virtualenv provides a safe and reliable mechanism for installing and running TensorFlow.

Take the following steps to install TensorFlow with virtualenv.

 

Step 1: Install pip and virtualenv by issuing following commands:

TensorFlow Step 1

Step 2: Create a virtualenv environment by issuing the following commands:

The preceding source command should change your prompt to the following:

Tensorflow step 2

Step 3:

Activate the virtualenv environment by issuing one of the following commands:

The preceding source command should change your prompt to the following:

TensorFlow step 3

 

Step 4:

Issue one of the following commands to install TensorFlow in the active virtualenv environment:

TensorFlow step 4

 

No module named tensorflow

Activate TensorFlow

Note that you must activate the virtualenv environment each time you use TensorFlow. If the virtualenv environment is not currently active, invoke one of the following commands:

When the virtualenv environment is active, you may run TensorFlow programs from this shell. Your prompt will become the following to indicate that your tensorflow environment is active:

active tensorflow environment

 

Deactivate TensorFlow

you may deactivate the TensorFlow environment by invoking the deactivate function as follows:

Run TensorFlow program

Enter the following short program inside the python interactive shell:

The preceding source code should change your prompt to the following.

TensorFlow Hello World

 

 

Related Post

Introduction TensorFlow Machine Learning Library

TenserFlow Lite

Train Image classifier with TensorFlow

Train your Object Detection model locally with TensorFlow

Android TensorFlow Machine Learning