Interactive Application for Market Analysis

Our client conducts market analysis for products by pulling posts/comments from social media networks such as Facebook, Twitter, YouTube, and Instagram. These analyses help companies evaluate how well their products are received on social media. Leveraging technologies such as ElasticSearch (for superior text search indexing) and Sencha Ext JS library (for agile development), we delivered a custom solution to the client. The application developed was visually appealing, interactive, fast, and one that could scale easily to accommodate the ever-growing dataset. QBurst was granted full product ownership for this solution.

Prior to engaging QBurst, the client used an application that retrieved information from social networks into their local PostgreSQL database. Their analysts worked on this data and generated reports to forecast market trends of various products. The client wanted QBurst to improvise this application by:

• Incorporating visualization capabilities
  • Live display of tweets in a map as and when they appear
  • Interactive graph highlighting retweets and shares as they appear in social media networks
• Improving application speed
• Incorporating scalability to accommodate the increasing dataset

The main challenge for QBurst was to figure out how to make the application faster and accommodate the ever growing dataset. We had to explore various JavaScript frameworks to choose the best that could incorporate changes easily and fast.

Primary study of the client’s system showed that they were using a relational database and doing heavy joins to fetch data. The growing database would eventually increase response time. We recommended moving to NoSQL for the following reasons:

• Joins can be avoided.
• Large amount of data can be stored.
• Results can be queried with faster response time.
• Transaction success (for example, missing of few tweets during insertion) would not be a factor in the final report as the application is more of a statistical data analysis.
• Fewer insertions/updates into the database while allowing for greater permutation of selections on the dataset.
• With increase in data, nodes can be increased to distribute data across machines.

• Our solution reduced the application load time immensely and ensured a faster response.
• The technology used guarantees a rich and aesthetically designed UI and faster loading up of visualizations.
• The solution renders a better overview of tweets with map visualization to study where majority of tweets come from and how they get re-tweeted.
• Map illustration gives a picture of whether tweet sentiments are positive or negative.