Have you ever wondered if there are signals in tweets that could be used to predict commodity futures? Farmers today use high tech equipment with rich sensors to monitor the state of their farms. Some of them even tweet about their crop conditions. Prominent news sources also share information about crop conditions and expected crop yields for the current agricultural year. I presented the proof-of-concept NLP pipeline for predicting commodity futures using tweets that we built for one of our customers in the agri-business space, at Text Analytics World 2013 in San Francisco.
The motivation behind this project we delivered for one of our customers were two fold:
- Demonstrate the value in harnessing a massively parallel processing database like Greenplum and power in-database machine learning libraries like Apache MADlib to solve business problems for our customer.
- Demonstrate the text analytics capabilities in Apache MADlib and PL/Python on Greenplum by analyzing tweets to predict commodity futures that were of interest to our customer.
You can view my slides from this talk below.
