IBM Watson & Cleversafe: Video Autotagging Analytics and Object Storage
Our BlueChasm development team came up with a pretty interesting prototype app, codenamed "Video Recon," that combines video analytics via IBM Watson and multiple layers of the IBM digital cloud stack. More specifically, our video analytics app uses IBM Watson Visual Recognition in IBM Bluemix in order to "look" at video files and automatically tag those video files with what Watson "sees" in the videos. Watson achieves this through advanced machine learning of images it "sees" and is trainable by the user to improve its accuracy over time.
The uploaded video and its associated tags are then stored in IBM Cleversafe Object Storage (located on-premises in our lab at Mark III) for further analysis and retrieval in the future by both current analytics platforms that we have and those that we have not yet thought to use. The tags generated by Watson serve as key metadata that will enable searches, filtering, and other forms of analysis by the user and other applications/services that may be seeking compelling insights from the large repository of videos stored on Cleversafe.
Before we dive into some screenshots of the actual app, here are some quick highlights of how our team built the app:
- Unique algorithm: The app itself actually autotags videos by using a unique composite scoring algorithm that takes the video and slices it up into images (each image is then "scored" by Watson and compiled into a final "video score" by our algorithm)
- Autotag and Cleversafe: The app itself utilizes IBM Watson in IBM Bluemix in the public cloud and stores the videos and its tags in IBM Cleversafe Object Storage on-premises in our lab
- Hybrid: Although IBM Secure Gateway is typically the way we would securely achieve this hybrid cloud scenario, we actually put together a one-off approach that utilizes Statica, among other tech
- Demo iPad App: For demo purposes, we wrote our own iPad app for Video Recon (screenshots below), which is currently in the hands of our field sales and engineering teams to be shown to our clients and ecosystem partners
- API on the way: Our intention is improve the app and possibly offer it as a service via APIs at some point, if there's any interest from the ecosystem
- Cleversafe is key: Although this app does show the power of IBM Watson in video analytics, we actually love more how it showcases IBM Cleversafe as an elite object storage solution and one that will power digital application use cases and the associated explosion of unstructured data for many years to come (in addition to solving many data challenges of current applications today, like data archiving and enterprise collaboration)
And here is how the Video Recon iPad app actually looks as of today (v1):
(To watch a recorded demo please click here!)
The user can either record his own video, upload an existing one from the iPad, or choose a sample video that has been preloaded:
For the purposes of this demo, we'll choose the preloaded video of a freight train (video on the right, which is around 10 seconds long in duration):
Video is then uploaded, which takes a few seconds:
Our Video Recon app then works in conjunction with IBM Watson:
The video is then returned with its Watson-generated tags, which does accurately tag the video with the word "Train." The video and its tags are then stored in Cleversafe for future use.
For more technical details on how we put together the app, please check back for future blog entries from our dev team that will break down exactly how they created this service.
If you'd like to see a live demo or have questions, please feel free to reach out to us via Live Chat (bottom right corner) or via our Contact Us form or on Twitter/Facebook. Thanks for your interest!