Video topic extraction and transcription via Speech2Topic API

This is a quick tutorial on how to transcribe and extract topics from videos using the Yactraq Speech2Topic API in Mashape.

(Note: Yactraq is giving away $500 to the winner Yactraq category for this coming Mashape April Hackathon on Apr 6th, 2013.  We require a number of teams to submit to this category to unlock the prize :))

This is particularly useful if you’re trying to get context in the video, which can be used to query ad services to pull relevant content.

The process is very simple, we send a link (YouTube) of a video to be processed by the Yactraq API, and we get back a result with information on topics and transcribed words. This result metadata will indicate when in the video these topics and words occur.

Here’s a sample of the response from a processed video:

{
  "status": "processed",
  "topics": [
    {
      "complete": false,
      "ts": 60.04,
      "topics": [
        "phones best",
        "best top phones",
        "droid"
      ],
      "scores": {
        "droid": 0.74029145,
        "phones best": 1.41333943,
        "best top phones": 1.28944674,
        "phones new": 0.36857178,
        "android phone android phone": 0.5815354,
        "the top phones": 0.55621447,
        "samsung galaxy tabs": 0.34378811,
        "the phone android": 0.36527107000000003
      },
      "sentiment": 1.2714676450366273
    }, ...
  ...
}

And here’s what an app that consumes it looks like in action: http://www.screenr.com/BKJ7

Getting started on Speech2Topic API

Step 1:  Get a Mashape key.  All calls to any Mashape API requires a Mashape key.  If you haven’t done so yet, you can sign up for an account at http://www.mashape.com/signup.  You can get your Mashape key in your Dashboard.

Step 2:  Request for a Yactraq developer account.  We need the Yactraq account so we can make a proper request to process our videos.  To request for an account, follow the instructions here.

Step 3:  Submit a video processing request through the Yactraq Speech2Topics API.  To do this, you would need the YouTube url of the video you want to extract topics from.  Make a note of the video id of it as well (e.g. http://www.youtube.com/watch?v=zFOnOLXrMBQ, the video id is “zFOnOLXrMBQ”).  We also need information from our Yactraq account, particularly “secret” and “adset” which you can get by logging in to http://api.yactraq.com/ .  Also set “start” and “tx” to 1.

My request looks like below:

Yactraq Speech2Topic API Request

Yactraq Speech2Topic API Request

 

Then hit “Test Endpoint”.  If your request is successful, you would get a response like below:

Yactraq API Sample Response

Yactraq API Sample Response

 

Step 4: Get some coffee and wait for your request to be processed (According to Yactraq the waiting time takes about half or a third of your video length).

Then you can try to check the status by calling the same endpoint, only this time set “start” to “0” or leave it blank.  This will tell the API to get the results, instead of initiating a new request.

Check status by calling the same endpoint with start empty or 0

Check status by calling the same endpoint with start empty or 0

 

If your request is successful, you should get a lengthy response like the one I showed you at the beginning of this tutorial.  What you didn’t see there are the transcription results which look like this:

  "transcript": [
    {
      "w": "thanked",
      "p": 0.087,
      "ts": 0.02
    },
    {
      "w": "never",
      "p": 1,
      "ts": 0.67
    },
    {
      "w": "nintendo",
      "p": 1,
      "ts": 2.27
    },
    {
      "w": "an",
      "p": 0.996,
      "ts": 3.79
    },
    {
      "w": "ann",
      "p": 0.52,
      "ts": 3.99
    },
  ...
}

For both parts of the response, there are “ts” values per topic or word transcribed.  You can use this ts (timestamp in seconds) value to see where the words and topics occurred in the video.

I have created a sample app that shows how this can be done here. (the app in the video at the top).

We hope this will get you started in building your own apps that use the Speech2Topic API in Mashape.  Let us know if you have suggestions or comments for our next tutorial!