Expert techniques for architecting end-to-end big data solutions to get valuable insights