Note: The first three payment/milestones below fall under “Applying text algorithm again, part1”, and the second three under “Applying text algorithm again, part2”
The aim of this project is to improve a classification+clustering method developed in a previous project, based on some technical issues that were identified.
1. The team is going to send the freelancer a list of irrelevant words.
2. As agreed in the end of the previous project, the same overall method should be carried out, with the same tools. As before, we are going to try three different levels of tolerance for the algorithm. We are going to try five versions of algorithm+clusterization to deal with the irrelevant word problem:
i. considering only words which occur in less than 10% of entries to create the algorithm and clustering
ii. considering only words which occur in less than XX% of entries (another cut-off chosen after the results from iii.) to create the algorithm and clustering
iii. ignoring words from the list of irrelevant words #1 to create the algorithm and clustering
iv. ignoring words from the list of irrelevant words #2 to create the algorithm and clustering
v. a version combining the use of the best list of irrelevant words and the best frequency cut-off
(In all versions we use the tool to fix typos. For each version we test three tolerance levels.)
3. In each case, compute the silhouette score both on predicted codes and on clusters
4. The team will assess the results using the list of irrelevant words #1 and if necessary bring some modifications to the list for the algorithm and clustering to be re-run (version ii. using list of irrelevant words #2).
5. Once the algorithm and clustering are finalized: assign a predicted code to each cluster, by comparing of "mean cluster sentence" with all code descriptions (from initial learning dataset + additional codebook) to choose the best matching code description.
6. The project ends when the algorithm and the clustering perform in a satisfactory way.
The team will then receive from the freelancer the codes/tool allowing them to re-run the exact same algorithm and clustering in the future and adjust them if necessary.
Payment plan:
First $278 for 2i, including algorithm results, clusterization results, silhouette score, and predicted code assigned to each cluster
Second Same, for 2ii
Third Same, for 2iii
Fourth Same, for 2iv
Fifth Same, for 2v
Sixth $280 for all codes/tools necessary to re-run and adjust the algorithm, clusterization, silhouette score calculation, and code for assigning codes to clusters
My name is Mike and I’m from UK. I work with individual clients and also provide outsourcing services for a number of UK and USA based agencies. Your project description sounds interesting to me and I do have skills & experience that are required to complete this project. I can show you some examples of my work. Please contact me to discuss your project.
Data Analyst/Scientist with more than 6+ years of experience in SPSS, STATA, SAS, MINITAB. I have been doing descriptive and inferential statistics
Key Techniques are
Regression Model
Binary Logistic Model
Factor Analysis
Cluster Analysis
Neural Network
Parametric and Non-Parametric Test
Good in data visualization using Data mining technique like CRT,QUESTetc
Please refer my client's feedback ( 5 star rated) Kindly reach out to me for further discussions