How to be a good contributor in Tatoeba

Introduction

This article by Trang is a must-read for anyone who has already read the quick start guide and is serious about contributing to Tatoeba. The original version, written in 2010, is updated slightly below.

Here is an outline of how to be a good contributor:

1. Understand the context of the project

I started this project in 2006. The initiative was driven by my passion for language learning and frustration about not finding an adequate online dictionary. The project is focused on sentences. Sample sentences were (and still are) a very scarce resource. Please only add complete sentences if you are going to contribute.
I was alone on this project for some time. It was only three years later, in 2009, that other people (all computer science students) started to help me out by coding more features.
Tatoeba is NOT a commercial project. We're not a company, and we're not paid for doing any of this. It is is something that we're working on in our free time.
To be honest, we don't exclude the possibility of starting a company someday, but that is if and only if we have an innovative, coherent and ethical business model (yeah, good luck). Having ads everywhere and driving a lot of traffic, or forcing people to pay to access the data, is out of the question.

2. Understand how the corpus is structured

The corpus is structured not as a table but as a graph (in the computer science sense of the word). What does that mean? Well, imagine you had to extract part of the corpus and write it on paper. You might do something like this:

English	French	Spanish
My name is Trang.	Je m'appelle Trang.	Me llamo Trang.
How are you?	Comment vas-tu?	¿Cómo estás?
...	...	...

That's a table structure. There are rows and columns: a row contains sentences with the same meaning, and a column contains sentences with the same language. That's the first approach anyone might take, but that's NOT how the corpus is constructed.

Our corpus is set up like this:

Schema of a graph of sentences in English, French and Spanish

That's a graph structure. There are nodes and edges: each node represents a sentence, and each edge represent the link between two sentences. When two sentences are linked, they have the same meaning.

The structure of the representation has a big effect on the way you can contribute to the corpus. One important implication of the graph structure is that you can add multiple translations in the same language for a specific sentence. You think there are two ways to translate a sentence and you really can't decide which would be the best? Well, just add both!

Some other implications are pointed out below.

3. Focus on the main sentence, not the other translations

When you translate a sentence, you are in fact adding a sentence (node) and a link (edge) between the original sentence and your translation. The only thing you need to care about is that you are adding a proper translation to the "main sentence" (the one at the top, in larger type).

Assume you wanted to add a Spanish translation to the English sentence "How are you?", which happens to have an existing French translation "Comment vas-tu ?" You would see the following:

How are you?

=> Comment vas-tu ?

You could add "¿Cómo estás?" (informal) or "¿Cómo está usted?" (formal). Even better, you could add both as separate translations of the same original sentence. You should not restrict yourself to choosing an informal translation simply because one was chosen by the person who was working in French. You should only care that your translation is a proper translation of the English sentence: if someone had to translate your contribution back to English, "How are you?" would be a possibility.

4. Translate the sentence as a whole rather than as a collection of individual words

We are not interested in having sentences that sound like they were written by a primitive robot that translated the individual words without regard for the sentence as a whole. We want sentences that a native speaker would really say. Translating is a very difficult task, we know. But if you are translating into your native language, you should always, always reread your translation in isolation from the original, and ask yourself if it is actually something people would say. If, for some reason, you want to add a word-by-word translation that is not what a native speaker would say, you should use the comments.

You are allowed to translate into a language other than your own, in which case you are forgiven for not writing native-like sentences. But in this case, please add a comment or tag to request that a native speaker check your sentences and correct any mistakes.

Tatoeba is not only about providing translations. It's also about gathering data on a language. Even if none of the sentences in a given language has yet been translated, those sentences are informative in themselves. However, they must be representative of their language.

To put it another way, the sentences are the basic layer in the Tatoeba corpus. The links between the sentences form another layer. However, the corpus should have value even without that second layer.

5. Do not edit a sentence if, by itself, it is correct

As I mentioned in the previous section, sentences in Tatoeba have value apart from their translations. Consequently, before you modify a sentence, look at it without paying attention to its translations, and ask yourself "Does this sentence have any spelling or grammar mistake? Does it sound weird?". If the answer is "No", then do NOT edit it! If one or more of the translations is incorrect, you should break the links between them (or ask someone else to do it for you), but the content of sentences that are valid in themselves should remain unchanged.

You may be tempted to edit a sentence so that its meaning matches all the other sentences. Or perhaps you want to turn a sentence into a word-for-word translation. But this is not a good idea. Not only does a word-for-word translation run the risk of sounding unnatural (cf. rule #4), but you're replacing data when you could be adding new data and keeping the old.

Sometimes, as the result of a mistake, a sentence may not match the original AT ALL. For instance:

My name is Trang.

=> Je m'appelle Trang.

=> Vamos a la playa.

You notice that the Spanish sentence (which says "Let's go to the beach") has nothing to do with the English sentence.

Perhaps you don't speak Spanish very well so you're not confident in modifying the Spanish sentence and decide to change the English sentence instead. However, then you create another problem: the French sentence won't fit the English sentence anymore...

Perhaps you are a native Spanish speaker and are tempted to change the Spanish sentence. In this particular case, it would still be acceptable because the Spanish sentence is not linked to any other sentence. But if someone had translated that Spanish sentence into Italian, "correcting" the Spanish sentence would cause a conflict with the Italian translation.

Then there is a problem you may have not thought of: when changing the meaning of a sentence, you are potentially erasing unique vocabulary. What if the Spanish sentence was currently the only one with "playa" in it?

The best way to proceed in this kind of situation is to add a new Spanish translation (Me llamo Trang) and "unlink" the current Spanish translation. NOTE: Only "trusted users" (users with a level of advanced contributor or higher) can unlink. However, anyone can post a comment to request that a sentence be unlinked.

6. Do not change the language in which a sentence is written

While you should correct the language flag for a sentence if it is wrong (for instance, it is flagged as Chinese when it is in fact Japanese), you shouldn't replace, say, a Japanese sentence by a Chinese sentence with the same meaning.

The problem is that a sentence can be associated with data, such as comments, that depend on its language. People can post comments on sentences, and the comments may be valid only because the sentence was in a certain language.

At the moment, this is mostly an issue for Japanese sentences, which are associated with special annotations. These annotations are not displayed because they are not useful for normal users. If you change a Japanese sentence into an English sentence, then the annotations that were associated with it won't make sense anymore.

7. Make sure you are adding comments to the right sentence

When you post a comment, the comment is only associated with the main sentence, so make sure that your comment is related to that particular sentence. Typically, imagine that you want to point out a spelling mistake, as in the following:

My name is Trang.

=> Je m'appel Trang.

=> Me llamo Trang.

You can see that the French sentence is wrong. It should be "appelle" and not "appel". If you posted your comment here, it would be associated with the English sentence (which is the main sentence, displayed at the top). This is not what you want. The right thing to do is to click on the French sentence first. It will change the display into:

Je m'appel Trang.

=> My name is Trang.

=> Me llamo Trang.

And then you can post your comment.

Now there is the case where you want to point out that a translation is wrong. Your comment will be related to two sentences, so where should you post it? Well, ideally, for this type of situation, there should be the possibility of commenting on a link between two sentences. But we don't have that, so we can only comment on a sentence. You are free to decide where you want to post your comment. Just remember that your comment must be related to the main sentence.

8. Do not add sentences from copyrighted content

We are distributing the corpus under the Creative Commons Attribution (or CC-BY) license. This makes it possible for anyone to re-use this data in any way they want as long as they mention Tatoeba in their work.

As a contributor, you have agreed with the terms of use, and therefore you are providing your contributions under the CC-BY license as well. This means we can reuse your data in any way we want as long as we mention you, which we do via the logs and statistics.

But providing your work under CC-BY means you also have some responsibilities for what you provide. And you have to know that you cannot legally redistribute data if it was copied from a source that doesn't clearly state that you can. Typically, you cannot (legally) copy all the sentences from a textbook and add them into Tatoeba.

Don't worry, you (and we) won't land in jail and be in debt for life if you've added a couple of sentences from a textbook. But the law forbids us from taking a significant amount of someone's work and reusing it without their consent. Producing sentences and translations is work, so be careful where you get the sentences from. Preferably, come up with your own sentences or take them from books that are in the public domain.

If you have added or seen sentences that were copied from a copyrighted source, change a few words so that it won't be exactly the same sentence. Or go negotiate with the authors and convince them to release their work under the CC-BY license so we can re-use it.

Please follow these guidelines so that we don't get sued.

9. Do not insert annotations into sentences

We want sentences to remain as natural as possible, so please do not insert annotations into them. For example we do NOT want sentences like these:

I (female) am happy.
It's raining cats and dogs. (idiom)
I like her/him.

Regarding sentences 1 and 2, if you need to indicate that a sentence is a proverb or female speech or whatsoever, then post a comment about it (or tag it, if you are a trusted user), but please do NOT add this information directly in the sentence.

Regarding sentence 3, instead of having only one sentence, split it into two sentences. Remember, you have the right to add multiple translations in a same language. So the following is encouraged:

Je l'aime bien.

=> I like her.

=> I like him.

There are various reasons why we don't want annotations.

They can be a problem for people who are using our data in order to improve a natural language processing system, for instance.
Your translation can be retranslated into another language, and it's less easy for people to translate sentences that contain alternatives (like "him/her"), since they may result in changes to other parts of the sentences, making the result unwieldy.
If we want to record audio for the sentence, we will need to choose what exactly to record, and annotations don't help.

10. Give us feedback

We know that Tatoeba is not perfect, so don't hesitate to tell us what you think is missing (though it is a good idea to see whether the subject has already been discussed on the Wall already). Also tell us if you see any spelling mistake, feel that some explanations are not clear, or encounter bugs.

We also know that Tatoeba is a cool project so feel free to tell us you like it too :P

11. Do not wait for us to code it if you can code it

As much as we welcome feedback, we welcome even more INITIATIVE. There are just sooo many things we could do. We can't take care of everything.

For instance we are distributing the entire corpus, but many people probably don't need all the sentences in all the languages. You may just want the English-Spanish sentences. Well instead of asking and waiting for us to provide a file with only English-Spanish sentences, you can code a tool (and please, tell us if you do) that will extract only what you want from our files.

That's just one example but if you are a programmer, there could be many things you could do yourself instead of waiting for us to do it. But of course, tell us so we don't duplicate your effort.

You also have to know that we are actually open source (under AGPL license) but we are not really "promoting" this aspect because:

The code hasn't met my standards of elegance yet... Still too many parts that make me cringe when I look at them.
We still don't have a sound methodology and organization in our way of working and I really don't have time to manage more people. However if you love the project and are really motivated to join the development team, then feel free to contact us =). See our introduction for new developers.

12. Indicate your languages in your profile

For people who didn't know, you can edit your profile by clicking on your username (at the top, in the menu bar).

Since Tatoeba involves languages, it can be very useful for other users to know which languages you can speak and how well you can speak them. In your profile, be sure to fill out the section immediately under your personal description. It contains an "Add a language" button. For each language you add, you can indicate your proficiency. Tell other users to indicate their languages as well (if they haven't already), especially if they have already contributed.

13. Encourage and educate new or old contributors

Community is very important in a project like Tatoeba. We just can't fulfill our ambitions without a strong community. But how do you build one? Well, one thing is NOT to make new users feel lost and isolated.

Part of this depends on the system. It has to be designed in a way that not only enables but also encourages users to interact with each other. Tatoeba offers a Wall, private messages, and comments.

And, of course, the other part depends on the community itself. It must make an effort to build its strength. If someone is asking a question to which you can answer, don't hesitate to help out. If you notice someone is doing something wrong, don't hesitate to (politely!) tell them the right way to do it. If you notice someone or some people have been contributing significantly, don't hesitate to drop a line (in a private message or on the Wall) to say "congratulations" or "thank you" for their work.

More generally speaking, if you have any idea on how to make Tatoeba a more socially pleasant place to be, then go ahead!

14. Spread the love

Last but not least: you love the project, we love the project, we all want this project to become the greatest language tool of all time, so bring more people into this adventure!

In the end, anyone who knows how to read and how to write can participate. There's no need to be a polyglot. Even if you can "just" hunt for mistakes and correct them or point them out, this will be already extremely helpful. If you have programming skills, you can be helpful in working with our software. The more people we have, the more mistakes we can fix, and the more data we can produce that people can rely on. And everyone can live happily ever after.