On Monday we had a meeting with Utopian where we discussed a draft for the new questionnaire. We discussed the questions and some general points regarding our translation category. Many LM did not make it to the meeting so we will distill in this post the major points made and we will propose a new draft of the questionnaire following the suggestions we received.
The main comments our draft received were:
- The score of the questionnaire did not start from a value of 100. For the sake of discussion we decided to work with positive scores and we think it may be worth doing so until we reach a consensus on the questionnaire. Before implementation, the scores can be adapted to follow the same structure that the devs are using.
- There should not be any distinction between major and minor mistakes, a mistake is a mistake. We had two questions regarding mistakes, one was dealing with major mistakes and the other was about minor mistakes. A major mistake is considered a mistake that changes the meaning of the sentence and a minor mistake is considered something that does not have as much influence. The two questions gave more granularity and fairness to the questionnaire in our view. However, nobody spoke in support of this during the meeting and we were asked to combine both questions into one.
- Add a question about legibility of the translated text. A translator could be accurate in his translation but choose a difficult word that not many users understand. This question will ensure that translators that care also about the legibility of the text will receive a higher score.
- We had a question about translation style and were were asked to change the word from "style" to "consistency"
- Merge all questions regarding contribution post grammar, style and descriptiveness into one. There was quite a bit of discussion over this, users debated whether contribution posts are important or not and if this will have sense in Utopian v2. But for now we rely on contribution posts and LM will need to receive enough information to carry out their review. On the other hand, a translator should be able to write without making grammar mistakes.
Here we propose two possible distributions of scores throughout our questions. In Option 1 we have some emphasis on the descriptiveness of the contribution post and the grammar and lower emphasis on volume and consistency of the translated words.
In Option 2 we have more emphasis on volume and consistency and descriptiveness and grammar of the contribution post have very little weight and serve just to give more granularity to the score.
Which of these two options do you think is better?
During the meeting, Utopian has also shown openness regarding suggestions that may require work from a developer's point of view. Right now their devs are quite busy but they are willing to discuss our proposals. Something we also agreed on is to link the reward for the LM to the volume of words their had to review. If you have more suggestions feel free to add them in this list:
https://docs.google.com/document/d/1EwCP8GWyoWdKTfh7jcqTJ1z8Va_wR02PqUReopYjFkE/edit?usp=sharing
For proposals regarding the questionnaire please use the comment section below. As usual, if you express your opinion on something we would also appreciate if you provide a counter proposal.