Criteria grid for the project
The table below contains the main criteria I will use to grade your project. Of course these are not “fixed and rigid”: you may have explored options that are not in the table. In this case, I will of course add them (positively) to your evaluation.
For this project, the related work section should be like a (short) literature review. You could look at paper(s) in a similar task but on different dataset. The goal is not to list all related work. Choose one or two interesting neural approaches you found in the literature, but which may be too complex to implement as part of this project. Note that you should probably focus on papers who appeared the the following scientific conferences:
- Natural Language Processing: ACL, NAACL, EACL, EMNLP, Coling
- Machine Learning: NIPS/NeurIPS, ICML, ICLR
All the papers published in the proceedings of these conferences can be accessed freely. The ACL anthology is an archive of all scientific NLP conferences/journals related to the Association of Computational Linguistics. You can use Google scholar to search for related work.
|Introduction||Short and brief introduction without motivation, examples, ...||Clear statement of the problem, motivation for the task, examples from the datasets, summary of the work.||+ critical remarks on the problem, "historical" context|
|Dataset||Short and brief exposition of the dataset.||In depth explanation of data annotation, information about manual annotation or scrapping method (pre-processing? filtering?).||+ data statistics and analysis, discussion about data bias.|
|Neural architecture||"We used a BiLSTM with Glove embeddings followed by a MLP."||Clear mathematical description of the neural network, clear identification of the hyper-parameters.||Architecture update proposition.|
|Experiments||Table with results.||Clear description of the different experiments, justification of different neural architectures/hyperparameters you choosed to explore, analysis of the results.||+ identification of the limits of these experiments, analysis of problematic instances in the dataset, adversarial examples.|
|Related work||No related work.||Non-neural approaches for this problem, pros/cons between neural and non-neural methods for this task.||Explanation of more complex architectures that have been proposed in the literature for this task (e.g. interpretable architectures...)|
|Code||Un-documented code||README file with instruction to use your program, comments and clear structure of the code.||Appendix with information about the main problem you encounter for implementation (functions that where difficult to code, code optimization you achieve to improve execution time, ...)|
|Conclusion||No or brief conclusion.||Possible extensions, open-ended questions regarding the problem/task, etc that results from your observation.||-|
|Layout||-||Figures numbering, legends. Citation in a valid format...||Self contained figures, coherent report structure.|