Case Study Goals

Collect and analyze events/logs data from virtualized data center to root cause issues with Compute, Storage & Network nodes and provide Predictive Analytics using :

  • Real time Analytics
  • Batch data processing
  • Machine Learning
  • Data visualization
  • Cloud based AWS platform was used for Deployment

Case Study Metrics

Collect and analyze events/logs data from virtualized data center to root cause issues with Compute, Storage & Network nodes and provide Predictive Analytics using :

  • Developed Analytics Solution for Virtualized Data Centers
  • 100k messages / sec & 5 TB data per day
  • More than 10 nodes in Hadoop and Spark cluster
  • Sharded MongoDB cluster
  • 8 node OpenTSDB cluster
  • 7 nodes of Elastic Search
  • 5 node Redis cache cluster
  • 5 node Graph DB – OrientDB

Case Study Implementation

Implementation

  • Data ingestion
  • Data storage / warehousing
  • Real-time pipeline
  • Analytics on Spark streaming clusters
  • Machine learning
  • Elastic search
  • Graph processing
  • Data visualization

@scale

  • scalable architecture & design
  • Big Data technology choices
  • scaling infrastructure
  • fault tolerance & reliability
  • caching

Solution Architecture

This architecture details all the working components and layers.

Convert your idea into reality