Version at: 26/03/2020, 11:26

# User experience test

Test performed by gillux on the 10/03/2020.

## User background

* Nationality: Brazilian
* Languages: Portuguese (native) and many other languages
* Job: private teacher of English/Spanish/Portuguese (and teaching computers to kids)
* Age: 33

R\. was born in Brazil and now lives in Rio de Janeiro. He was first introduced to Tatoeba about 6 years ago by Shishir. He and Shishir used to study languages together on another website, but that website was bought by a company who "just wanted to make more money".

On Tatoeba, R\. translates from English/French/Spanish to Portuguese, studies language on his own. From about 2017, R\. started to add languages using Tatoeba’s Github. He is currently corpus maintainer and taking care of new language requests. His has a list of 74 languages on his profile page and he is "working on making it bigger and bigger everyday".

R\. is very happy and enthusiastic about Tatoeba because it allows him to study languages, talk to people, ask for help and contribute Portuguese sentences of various type (formal, informal…).

R\. doesn’t teach in a school because schools in Brazil don’t pay so much. Instead, he gives private classes on Skype, in the student’s home, at the library… R\. loves his job.

R\. praises Tatoeba because it doesn’t consist of robot added sentences, but sentences added and proofread by real people, every single day. Mistakes eventually get corrected because people are looking and commenting.

R\. introduces Tatoeba to his students by explaining they have to study languages using real life sentences, not word by word or using definitions. Not using theories but in practice. "Sentences are of high quality, they get proofread all the time, people work really hard on this, with so many languages to study."

About the quality of sentences contributed by native speakers: "Not being a native doesn’t mean your sentence is bad. And the opposite is true: I sometimes commit mistakes in Portuguese too." "Being a native of not is not important, what’s important is whether you have the knowledge, whether you can speak the language."

## Test

The test was performed remotely using Jitsi Meet and desktop sharing, but the desktop sharing was not working most of the time.

* OS: Windows
* Browser: Chrome
* Instructions: I asked R\. to show me how he uses Tatoeba as a corpus maintainer, as a language learner and as teacher.


### Using Tatoeba as a corpus maintainer

R\. explained that he performs the actions in this section at least once a week.

* R\. goes to his profile page where he keeps important links in his profile description.

#### My profile

* R\. clicks on the link @change

#### Sentences tagged as @change

* R\. explains he can filter by Portuguese and check whether the sentences are okay or not. He’d go to the sentence page to check if anybody left a comment.
* R\. shows another way: he clicks on "My ratings" from the top-right user menu.

#### My ratings

* R\. explains he’d check the sentences marked as "not ok".
* R\. adds that he’d actually got to "outdated" first and he clicks that link.

#### My ratings - Outdated ratings

* R\. explains he’d use that page to check if the sentences were corrected. If they are not, and it’s been more than 15 days, he may fix them.
* R\. decides to check a sentence that has an outdated rating.

#### Sentence 8529831

* R\. removes his outdated review.
* R\. goes back to the previous page.

#### My ratings - Outdated reviews

* R\. explains he’d keep doing this until he finishes the outdated list, and then go to sentences marked as "not OK" and do the same.
* R\. clicks on the link Sentences marked as "not OK"

#### My ratings - Sentences marked as "not OK"

* R\. identifies a sentence that is not a sentence and clicks it.

#### Sentence 8496443

* R\. deletes the sentence.
* R\. leaves a comment on the sentence.

### Using Tatoeba as a language learner

I asked R\. to show me how he’d use Tatoeba to learn a new language or study.

* R\. clicks on *Browse by list* from the top menu.

#### All public lists

* "First of all, I let Tatoeba know. I write here the language I study or what I’m studying in a certain language."
* R\. creates a list called "French sentences"

#### List "French sentences"

* R\. clicks on Translate sentences from the Contribute menu

#### Translate sentences

* R\. selects sentences in French, no directly translated in Portuguese, random.

#### Advanced search

* R\. explains that he’d add sentences to his list from that page. He’d add sentences that he does not fully understand, to later do some more research, to ask around for the meaning. He’d ask to native speakers, French speakers living around, French speakers on Tatoeba by PM, to Facebook contacts, by email or to Whatsapp contacts.
* For example, R\. can fully understand "Il cherche un appartment.", so he wouldn’t add it to his list.
* R\. reads the sentence "Je ne voulais pas que vous vous sentiez seul.", which he can also understand, but he’d like to better understand the word "voulais", so he’d add it to the list.

## Debrief

* R\. wishes to have a kind of forum on which he could ask about the meaning of a sentence when he’s studying (for example: is it a kind of slang? is it said only in France? only in Paris? only in Canada? what’s the real meaning of this sentence? why is this sentence "built" in this way?). R\. doesn’t feel comfortable asking on the Wall, and he doesn’t feel comfortable putting a comment on the sentence, because members may answer in French (which he doesn’t fully understand), or because some admin might complain that Tatoeba is not a place to study, that he should focus on his native language, etc. so R\. prefers to "keep this secret".

* R\. wishes to be able to choose "has audio" and the sort order directly from the top search bar because he uses it very frequently. Right now, he needs to click on Advanced search to do that.

* I asked R\. how he actually uses the lists that he created in preparation for his classes during his classes (whether he opens Tatoeba, downloads them as CSV etc.)

    * R\. downloads the list in L2 as CSV, eventually along with their translations in L1 for the students. He opens the CSV file in Excel and prints out the sentences, so he can use them during his classes. During a class, he cannot use the Tatoeba website directly because often there is no computer available or even no Internet connection. This workflow is satisfying for R\., but it could be more convenient for him to have the list in a PDF file. R\.’s students are not used to CSV files, so when R\. sends them a CSV file, they don’t know how to open it or suspect it’s a virus or something. PDF or DOC are more friendly.

    * Having the list as a DOC file would be useful because he can freely edit the text. Currently, he has to copy and paste sentences from the CSV file into a Word document to edit them the way he wants. When R\. teaches to kids, he likes to add some pictures or colors to the sentences. For example, he puts English sentences in red and Brazilian sentences in green. Otherwise, he writes in the Word document the usernames he got the sentences from, the link to the list (so the students can check later by themselves, and it kinda stimulates them to use Tatoeba).

* Every single time R\. makes a search, he has to check the "Help" link in the top search bar because he doesn’t remember the search operators like ^ $ -. He wishes there would be some hints to quickly know the basic features, for example, how to find:
    * sentences that begin with the word "..."
    * sentences that end with the word "..."
    * sentences that contains the word "..." but not begin or end with it
    * sentences that are exactly like "..."

* The "Native speakers" page is just a list with numbers, and R\. would like to have a link to easily *meet* these users. For example, on the first line English, there are two admins, four corpus maintainers, fifteen advanced contributors and so on. But R\. cannot talk or leave a message to any of them. It is frustrating for R\. because if he has a question for them, he has to find another (less quick) way.

* The same goes for "Languages of members". There is a link for each language, that shows people speaking that language. But from that list, there is no way to know if a single user is active or not (the last time he posted a sentence or left a comment or correct something). Even if R\. knows if a given user is active, it takes too much steps to contact the user (click on username, unfold the user menu and click on the contact button).

* The same goes for the "List of all members" page. You can’t tell when was the last time a user used Tatoeba, if a user reply to private messages, if you can rely on the user or not, if the user can solve a problem for you or not. For example: I have to contact an admin now because there is a fight. Which of these users can I contact? I don’t know.

* Let’s say I want to check users who posted or edited sentences in a given language. I expect to be able to do that from the Community menu, but I can’t. Instead, I have to go to the home page, scroll to the Latest contributions, click Show more, and then select a language from the drop down. This is not intuitive. From that page, if R\. sees that a given language is not very active (for example Korean, last posted sentence 8 days ago, or Uzbek, 27 days ago), he knows he cannot rely on Tatoeba to learn this language. From that page. R\. wishes he can contact a user directly instead of having to go through the profile page.

### Using Tatoeba as a teacher

* "Let’s suppose my class will be about the weather."
* R\. clicks on *Browse by list* from the top menu.

#### All public lists

* R\. tries to clicks on the Name field while the page is still loading, but the form jumps down when the page is fully loaded, so he misses it.
* R\. creates a list called "ENG - Weather"

#### List "ENG - Weather"

* R\. performs a search about the weather from the top search box.

#### Search results (from=English, to=none, keywords=hot|cold|forecast)

* R\. points out that the first result "Hot outside?" is interesting because people are used to the traditional English, not the English on the street for example. So he’d add it to his list.
* R\. points out at the third result "It’s cold." because it has audio, so his students can know how to pronounce the sentence correctly, improve their listening and their speaking.
* R\. points out at the result "I feel cold." and explains that he sometimes take notes in his notebook or a sheet of paper. While that sentence is not related to weather, but it’s a very common way to talk about the weather. So he can explain to his students that they can say "I feel cold." to mean "It’s cold."

## Identified problems

* Tatoeba doesn’t match the user needs:

* Usability problems:

* Feeling problems:

version at: 26/03/2020, 11:29

# User experience test

Test performed by gillux on the 10/03/2020.

## User background

* Nationality: Brazilian
* Languages: Portuguese (native) and many other languages
* Job: private teacher of English/Spanish/Portuguese (and teaching computers to kids)
* Age: 33

R\. was born in Brazil and now lives in Rio de Janeiro. He was first introduced to Tatoeba about 6 years ago by Shishir. He and Shishir used to study languages together on another website, but that website was bought by a company who "just wanted to make more money".

On Tatoeba, R\. translates from English/French/Spanish to Portuguese, studies language on his own. From about 2017, R\. started to add languages using Tatoeba’s Github. He is currently corpus maintainer and taking care of new language requests. His has a list of 74 languages on his profile page and he is "working on making it bigger and bigger everyday".

R\. is very happy and enthusiastic about Tatoeba because it allows him to study languages, talk to people, ask for help and contribute Portuguese sentences of various type (formal, informal…).

R\. doesn’t teach in a school because schools in Brazil don’t pay so much. Instead, he gives private classes on Skype, in the student’s home, at the library… R\. loves his job.

R\. praises Tatoeba because it doesn’t consist of robot added sentences, but sentences added and proofread by real people, every single day. Mistakes eventually get corrected because people are looking and commenting.

R\. introduces Tatoeba to his students by explaining they have to study languages using real life sentences, not word by word or using definitions. Not using theories but in practice. "Sentences are of high quality, they get proofread all the time, people work really hard on this, with so many languages to study."

About the quality of sentences contributed by native speakers: "Not being a native doesn’t mean your sentence is bad. And the opposite is true: I sometimes commit mistakes in Portuguese too." "Being a native of not is not important, what’s important is whether you have the knowledge, whether you can speak the language."

## Test

The test was performed remotely using Jitsi Meet and desktop sharing, but the desktop sharing was not working most of the time.

* OS: Windows
* Browser: Chrome
* Instructions: I asked R\. to show me how he uses Tatoeba as a corpus maintainer, as a language learner and as teacher.


### Using Tatoeba as a corpus maintainer

R\. explained that he performs the actions in this section at least once a week.

* R\. goes to his profile page where he keeps important links in his profile description.

#### My profile

* R\. clicks on the link @change

#### Sentences tagged as @change

* R\. explains he can filter by Portuguese and check whether the sentences are okay or not. He’d go to the sentence page to check if anybody left a comment.
* R\. shows another way: he clicks on "My ratings" from the top-right user menu.

#### My ratings

* R\. explains he’d check the sentences marked as "not ok".
* R\. adds that he’d actually got to "outdated" first and he clicks that link.

#### My ratings - Outdated ratings

* R\. explains he’d use that page to check if the sentences were corrected. If they are not, and it’s been more than 15 days, he may fix them.
* R\. decides to check a sentence that has an outdated rating.

#### Sentence 8529831

* R\. removes his outdated review.
* R\. goes back to the previous page.

#### My ratings - Outdated reviews

* R\. explains he’d keep doing this until he finishes the outdated list, and then go to sentences marked as "not OK" and do the same.
* R\. clicks on the link Sentences marked as "not OK"

#### My ratings - Sentences marked as "not OK"

* R\. identifies a sentence that is not a sentence and clicks it.

#### Sentence 8496443

* R\. deletes the sentence.
* R\. leaves a comment on the sentence.

### Using Tatoeba as a language learner

I asked R\. to show me how he’d use Tatoeba to learn a new language or study.

* R\. clicks on *Browse by list* from the top menu.

#### All public lists

* "First of all, I let Tatoeba know. I write here the language I study or what I’m studying in a certain language."
* R\. creates a list called "French sentences"

#### List "French sentences"

* R\. clicks on Translate sentences from the Contribute menu

#### Translate sentences

* R\. selects sentences in French, no directly translated in Portuguese, random.

#### Advanced search

* R\. explains that he’d add sentences to his list from that page. He’d add sentences that he does not fully understand, to later do some more research, to ask around for the meaning. He’d ask to native speakers, French speakers living around, French speakers on Tatoeba by PM, to Facebook contacts, by email or to Whatsapp contacts.
* For example, R\. can fully understand "Il cherche un appartment.", so he wouldn’t add it to his list.
* R\. reads the sentence "Je ne voulais pas que vous vous sentiez seul.", which he can also understand, but he’d like to better understand the word "voulais", so he’d add it to the list.

### Using Tatoeba as a teacher

* "Let’s suppose my class will be about the weather."
* R\. clicks on *Browse by list* from the top menu.

#### All public lists

* R\. tries to clicks on the Name field while the page is still loading, but the form jumps down when the page is fully loaded, so he misses it.
* R\. creates a list called "ENG - Weather"

#### List "ENG - Weather"

* R\. performs a search about the weather from the top search box.

#### Search results (from=English, to=none, keywords=hot|cold|forecast)

* R\. points out that the first result "Hot outside?" is interesting because people are used to the traditional English, not the English on the street for example. So he’d add it to his list.
* R\. points out at the third result "It’s cold." because it has audio, so his students can know how to pronounce the sentence correctly, improve their listening and their speaking.
* R\. points out at the result "I feel cold." and explains that he sometimes take notes in his notebook or a sheet of paper. While that sentence is not related to weather, but it’s a very common way to talk about the weather. So he can explain to his students that they can say "I feel cold." to mean "It’s cold."

## Debrief

* R\. wishes to have a kind of forum on which he could ask about the meaning of a sentence when he’s studying (for example: is it a kind of slang? is it said only in France? only in Paris? only in Canada? what’s the real meaning of this sentence? why is this sentence "built" in this way?). R\. doesn’t feel comfortable asking on the Wall, and he doesn’t feel comfortable putting a comment on the sentence, because members may answer in French (which he doesn’t fully understand), or because some admin might complain that Tatoeba is not a place to study, that he should focus on his native language, etc. so R\. prefers to "keep this secret".

* R\. wishes to be able to choose "has audio" and the sort order directly from the top search bar because he uses it very frequently. Right now, he needs to click on Advanced search to do that.

* I asked R\. how he actually uses the lists that he created in preparation for his classes during his classes (whether he opens Tatoeba, downloads them as CSV etc.)

    * R\. downloads the list in L2 as CSV, eventually along with their translations in L1 for the students. He opens the CSV file in Excel and prints out the sentences, so he can use them during his classes. During a class, he cannot use the Tatoeba website directly because often there is no computer available or even no Internet connection. This workflow is satisfying for R\., but it could be more convenient for him to have the list in a PDF file. R\.’s students are not used to CSV files, so when R\. sends them a CSV file, they don’t know how to open it or suspect it’s a virus or something. PDF or DOC are more friendly.

    * Having the list as a DOC file would be useful because he can freely edit the text. Currently, he has to copy and paste sentences from the CSV file into a Word document to edit them the way he wants. When R\. teaches to kids, he likes to add some pictures or colors to the sentences. For example, he puts English sentences in red and Brazilian sentences in green. Otherwise, he writes in the Word document the usernames he got the sentences from, the link to the list (so the students can check later by themselves, and it kinda stimulates them to use Tatoeba).

* Every single time R\. makes a search, he has to check the "Help" link in the top search bar because he doesn’t remember the search operators like ^ $ -. He wishes there would be some hints to quickly know the basic features, for example, how to find:
    * sentences that begin with the word "..."
    * sentences that end with the word "..."
    * sentences that contains the word "..." but not begin or end with it
    * sentences that are exactly like "..."

* The "Native speakers" page is just a list with numbers, and R\. would like to have a link to easily *meet* these users. For example, on the first line English, there are two admins, four corpus maintainers, fifteen advanced contributors and so on. But R\. cannot talk or leave a message to any of them. It is frustrating for R\. because if he has a question for them, he has to find another (less quick) way.

* The same goes for "Languages of members". There is a link for each language, that shows people speaking that language. But from that list, there is no way to know if a single user is active or not (the last time he posted a sentence or left a comment or correct something). Even if R\. knows if a given user is active, it takes too much steps to contact the user (click on username, unfold the user menu and click on the contact button).

* The same goes for the "List of all members" page. You can’t tell when was the last time a user used Tatoeba, if a user reply to private messages, if you can rely on the user or not, if the user can solve a problem for you or not. For example: I have to contact an admin now because there is a fight. Which of these users can I contact? I don’t know.

* Let’s say I want to check users who posted or edited sentences in a given language. I expect to be able to do that from the Community menu, but I can’t. Instead, I have to go to the home page, scroll to the Latest contributions, click Show more, and then select a language from the drop down. This is not intuitive. From that page, if R\. sees that a given language is not very active (for example Korean, last posted sentence 8 days ago, or Uzbek, 27 days ago), he knows he cannot rely on Tatoeba to learn this language. From that page. R\. wishes he can contact a user directly instead of having to go through the profile page.

## Identified problems

* Tatoeba doesn’t match the user needs:

* Usability problems:

* Feeling problems:

Note

The lines in green are the lines that have been added in the new version. The lines in red are those that have been removed.

Actions

• History
• Revert to version made the 26/03/2020, 11:26
• Go back to article's page