Angular2: Speech IMDB using Angular2, Typescript, Web API and ASP.NET MVC

Standard

Speech based Search Movie Web App using Angular2, TypeScript, BING API, ASP.NET MVC, Web API & Web Speech API

This web application assist you to search movies and seasons through voice. Just say any keyword from movie and season name and then command by saying term search to look from millions of record rapidly.

Project Website

Project Code


Dashboard

This is the homepage of application and displays latest Hollywood news at the moment. BING API has been used over here to retrieve latest Hollywood news.

screenshot_1


Manual Search

Just for the ease, manual search has been provided with parameter likes searching content name, year, content type(movie, series, episode). Please note year and type are optional.

screenshot_2


Voice Search

Awesome part starts from here, you may say any thing to search for movie, series and episode. You will also able to see on runtime in message box and console window what keyword your have voiced out to search.

screenshot_8

screenshot_9

screenshot_10

screenshot_11


Movie List Page

This page shows list of all movies, series and episode relevant to your search keyword, pagination has been implemented in case of large returned data set along with sorting functionality. Thumbnail associated with each record will help you to quickly pick your content.

screenshot_6


Movie Detail Page

This page shows full detail of user selected movie/series/episode. It displays unique url to checkout content’s reviews on IMDB , content’s rating, votes and full fledged poster.

screenshot_7

Advertisements

Angular2: Using Speech Recognition of Web Speech API in Angular2

Standard

Speech Recognition functionality of the Web Speech API in Angular2 application

Speech Recognition

Speech recognition involves receiving speech through a device’s microphone, which is then checked by a speech recognition service against a list of grammar (basically, the vocabulary you want to have recognised in a particular app.) When a word or phrase is successfully recognised, it is returned as a result (or list of results) as a text string, and further actions can be initiated as a result.

The Web Speech API has a main controller interface for this — SpeechRecognition — plus a number of closely-related interfaces for representing grammar, results, etc. Generally, the default speech recognition system available on the device will be used for the speech recognition — most modern OSes have a speech recognition system for issuing voice commands. Think about Dictation on Mac OS X, Siri on iOS, Cortana on Windows 10, Android Speech, etc.


Chrome support

As mentioned earlier, Chrome currently supports speech recognition with prefixed properties, therefore at the start of our code we include these lines to feed the right objects to Chrome, and non-prefix browsers, like Firefox:

var SpeechRecognition = SpeechRecognition || webkitSpeechRecognition
var SpeechGrammarList = SpeechGrammarList || webkitSpeechGrammarList
var SpeechRecognitionEvent = SpeechRecognitionEvent || webkitSpeechRecognitionEvent  

Properties

  • SpeechRecognition.lang: Sets the language of the recognition. Setting this is good practice, and therefore recommended.
  • SpeechRecognition.interimResults: Defines whether the speech recognition system should return interim results, or just final results. Final results are good enough for this simple demo.
  • SpeechRecognition.maxAlternatives: Sets the number of alternative potential matches that should be returned per result.
          recognition.continuous = false;
          recognition.lang = 'en-US';
          recognition.interimResults = false;
          recognition.maxAlternatives = 1;
    

Event handlers

  • SpeechRecognition.onaudiostartFired when the user agent has started to capture audio.
  • SpeechRecognition.onaudioendFired when the user agent has finished capturing audio.
  • SpeechRecognition.onendFired when the speech recognition service has disconnected.
  • SpeechRecognition.onerrorFired when a speech recognition error occurs.
  • SpeechRecognition.onnomatchFired when the speech recognition service returns a final result with no significant recognition. This may involve some degree of recognition, which doesn’t meet or exceed the confidence threshold.
  • SpeechRecognition.onresultFired when the speech recognition service returns a result — a word or phrase has been positively recognized and this has been communicated back to the app.
  • SpeechRecognition.onstartFired when the speech recognition service has begun listening to incoming audio with intent to recognize grammars associated with the current SpeechRecognition.

Method

  • SpeechRecognition.abort()Stops the speech recognition service from listening to incoming audio, and doesn’t attempt to return a SpeechRecognitionResult.
  • SpeechRecognition.start()Starts the speech recognition service listening to incoming audio with intent to recognize grammars associated with the current SpeechRecognition.
  • SpeechRecognition.stop()Stops the speech recognition service from listening to incoming audio, and attempts to return a SpeechRecognitionResult using the audio captured so far.

Starting the speech recognition

recognition.start();

Receiving and handling results

recognition.onresult = function(event) {
  var last = event.results.length - 1;
  console.log(event.results[last][0].transcript);
  console.log('Confidence: ' + event.results[0][0].confidence);
}

The SpeechRecognitionEvent.results property returns a SpeechRecognitionResultList object containing SpeechRecognitionResult objects. It has a getter so it can be accessed like an array — so the [last] returns the SpeechRecognitionResult at the last position. Each SpeechRecognitionResult object contains SpeechRecognitionAlternative objects that contain individual recognised words. These also have getters so they can be accessed like arrays — the [0] therefore returns the SpeechRecognitionAlternative at position 0

Stopping the speech recognition

recognition.stop(); 

Handling error in the speech recognition

SpeechRecognition.onerror handles cases where there is an actual error with the recognition successfully — the SpeechRecognitionError.error property contains the actual error returned:

recognition.onerror = function(event) {
  console.log(event.error);
}

Web Speech API in Angular2

See extended DEMO on Github

Angular2 Service using Web Speech API

import { Injectable, NgZone } from '@angular/core';
import { Observable } from 'rxjs/Rx';
import * as _ from "lodash";

interface IWindow extends Window {
    webkitSpeechRecognition: any;
    SpeechRecognition: any;
}

@Injectable()
export class SpeechRecognitionService {
    speechRecognition: any;

    constructor(private zone: NgZone) {
    }

    record(): Observable<string> {

        return Observable.create(observer => {
            const { webkitSpeechRecognition }: IWindow = <IWindow>window;
            this.speechRecognition = new webkitSpeechRecognition();
            //this.speechRecognition = SpeechRecognition;
            this.speechRecognition.continuous = true;
            //this.speechRecognition.interimResults = true;
            this.speechRecognition.lang = 'en-us';
            this.speechRecognition.maxAlternatives = 1;

            this.speechRecognition.onresult = speech => {
                let term: string = "";
                if (speech.results) {
                    var result = speech.results[speech.resultIndex];
                    var transcript = result[0].transcript;
                    if (result.isFinal) {
                        if (result[0].confidence < 0.3) {
                            console.log("Unrecognized result - Please try again");
                        }
                        else {
                            term = _.trim(transcript);
                            console.log("Did you said? -> " + term + " , If not then say something else...");
                        }
                    }
                }
                this.zone.run(() => {
                    observer.next(term);
                });
            };

            this.speechRecognition.onerror = error => {
                observer.error(error);
            };

            this.speechRecognition.onend = () => {
                observer.complete();
            };

            this.speechRecognition.start();
            console.log("Say something - We are listening !!!");
        });
    }

    DestroySpeechObject() {
        if (this.speechRecognition)
            this.speechRecognition.stop();
    }

}

Component using above service

import { Component, OnInit, OnDestroy} from '@angular/core';
import { SpeechRecognitionService } from './speech-recognition.service';

@Component({
    selector: 'my-app',
    templateUrl: './App/app.component.html'
})

export class AppComponent implements OnInit, OnDestroy {
    showSearchButton: boolean;
    speechData: string;

    constructor(private speechRecognitionService: SpeechRecognitionService) {
        this.showSearchButton = true;
        this.speechData = "";
    }

    ngOnInit() {
        console.log("hello")
    }

    ngOnDestroy() {
        this.speechRecognitionService.DestroySpeechObject();
    }

    activateSpeechSearchMovie(): void {
        this.showSearchButton = false;

        this.speechRecognitionService.record()
            .subscribe(
            //listener
            (value) => {
                this.speechData = value;
                console.log(value);
            },
            //errror
            (err) => {
                console.log(err);
                if (err.error == "no-speech") {
                    console.log("--restatring service--");
                    this.activateSpeechSearchMovie();
                }
            },
            //completion
            () => {
                this.showSearchButton = true;
                console.log("--complete--");
                this.activateSpeechSearchMovie();
            });
    }

}

Usage

Before using project Install dependencies by typing below command in CMD:

npm install --save --save-dev

Output

Initial State:

screenshot_19

Example1:

screenshot_20

Example2:

screenshot_21

Reference

Text Reference.

API Reference