text mining with python

---

شرح پروژه

This exercise consists of implementing the word2vec method together with Fuzzy Rough Nearest Neighbor (FRNN) algorithm to detect emotions in tweets. 

Classification description: 

We formulate the exercise as a classification problem where predicting data (X-data) are different tweets, while for a given emotion, labels (Y-data) are values expressing how much the emotion is present. Labels take values from the set {0,1,2,3} where 0 means that the emotion is not present, while 3 means that the emotion is fully present. We will consider four different emotions (anger, fear, joy, sadness), so four corresponding datasets will be provided.  

Exercise description: 

1. For given tweets, use word2vec methods from the python library gensim to obtain numerical representation of tweets. Describe briefly which method you used to perform word2vec. Be aware, the tweets are containing special symbols, so some preprocessing may be necessary. 

2. Now, when you have a numerical representation of a tweet, you have to construct a similarity relation R: tweets x tweets -> [0,1], which will, for two given tweets, tell you how similar they are on the scale [0,1]. 1 means they are identical, while 0 means that there is no similarity between them. Values between 0 and 1 express partial similarity. Discuss properties of such similarity relations. 

3. Using similarity relations constructed in the previous step, implement the FRNN method described in the article: Jensen, R., & Cornelis, C. (2011). Fuzzy-rough nearest neighbor classification and prediction. Theoretical Computer Science, 412(42), 5871-5884. 

4. Use 5-fold cross validation to evaluate the performance of the implemented method. Discuss which metrics can be used for the evaluation on the given data.

 5. Discuss possible improvements of the classification algorithm theoretically (you do not have to implement anything on this point). These improvements may refer to improving efficiency, accuracy, adapting the algorithm to this particular type of data, etc. 

Rules for the exercise: 

1. All programing work has to be implemented in Python in one single .py file. 

2. That file has to be runnable on a command: python your_file.py dataset.csv  and to return some cross-validation score in a form (mean, standard deviation). 

3. All theoretical discussions have to be provided in a single .pdf file. So, at the end, your submission will contain one .py and one .pdf file. 

4. External python libraries allowed to implement the task are genism, numpy and pandas. Using other external libraries is not permitted. 

5. Time efficiency will be examined too. Avoid using “for” loops and use the numpy commands as much as you can to improve efficiency. 

6. Experimental results have to be provided in the .pdf file. Provide cross-validation scores for different similarity relations and different metrics in the form (mean, standard deviation). 

این پروژه شامل 1 فایل مهم است، لطفا قبل از ارسال پیشنهاد حتما نسبت به بررسی این فایل اقدام فرمایید.

مهارت ها و تخصص های مورد نیاز

---

---