High performance mining of social media data

News and disaster-related applications may benefit from real-time processing of large-volume, up-to-the-minute social media data. Our geo-mining algorithm finds local place references (of street, building, toponym and place abbreviation) in Twitter messages so that those messages can be put on a map. The ability to map is significant because it can present a timely overview of a situation. Our current research demonstrates that our prototype desktop algorithm that geo-locates Twitter messages with an F statistic of .90 accuracy for location identification will be viable on a large scale and in real time, for actual applications. We present methods of managing external resources, threading the algorithm and balancing the data load, that allow us to scale up the application without significantly re-writing the code.