Tuesday, September 24, 2013

Big Data for Real Time Offers

Lily is its name. Lily can apply machine learning to measure attractive score for each offers based on large amount of data gathered for a particular customers. Lily's prowess enables NGDATA to raise $3.3M in its second round of funding. The article explains the mechanism on how the flagship product Lily can offer the best real-time offers to users. From the article, there are two important points worthy of discussion. One is privacy concerns and second is job opportunity, or rather job security.

For real-time offers to work, Lily needs to systematically collect real-time GPS locations from its users. It would also need to know other previously private data such as past credit card transactions, twitter feeds, text messages. These data will help Lily the system to estimate the relevance for an advertisement to be served to a particular customer at a given time.

For example, a customer is walking around central park at 2pm, and it was a hot day. He used his credit card to purchase a nice cold frappuccino from Peet's coffee on a previous hot day. The system would recognize that an advertisement would be relevant if it is from a store with cold refreshments. An example ad might be, "Grab a soft-serve ice cream from ABC store now! 15% off just for you in the next 30 minutes"

Some may find the ads relevant while others may be bothered about the Big Brother surveillance like program to serve ads. If any data is collected and retained by any company, it is only time before government demands it in the name of national security or other purposes. The prevalence of data poses great potential for advertisers, but it also poses great threat to personal liberty and the right to conduct one's life without being monitored. Privacy safeguards are nice, but can never be 100% effective. Only by not collecting such data can privacy breach be 100% eliminated.

Second point is about job security for the consulting services to implement the system. "Channels that Lily can tap up include data contained in subscriber databases and CRM systems; behavioural and transactional data including calls, texts, payments, iDTV, credit card transactions; contextual data such as weather, geo-location, or “inferable context” such as social sentiment or network data; logs from online and mobile applications; third party data such as socio-demographic data; and social media data." All these data are not uniformed and stored in separate storage silos. Integrating these data means consulting service opportunities. Few high level executives and the marketing professionals would even understand the difference between SQL and MapReduce. Using Lily means many many months and even years of data integration work. No wonder data scientists are some of the highest paid professionals among all engineers.

Inspiration comes from:
http://techcrunch.com/2013/09/24/ngdata-series-a/

No comments: