Version at: 31/01/2020, 18:13

# User experience test

Test performed by gillux on the 31/01/2020.

## User background

* Nationality: French
* Languages: French (native), English (advanced)
* Job: helper at a surf hostel
* Age: 28

S\. joined Tatoeba in 2012 and contributed for a short time. He hasn’t been using it much since. In 2012, he used Tatoeba to learn basic Chinese vocabulary. His main use case was the creation of Anki flashcards. S\. was manually copy and pasting sentences into a text editor to create CSV data for Anki. If S\. was about to use Tatoeba again now, he’d use it exactly the same way.

At some point, S\. downloaded the CSV exports, tried play around with it to avoid copy and pasting things. However, as S\. is not a developer, he didn’t manage to complete the script he wanted to automate his process.

S\. is much interested in collaborative projects like Wikipedia. He values Tatoeba because of its philosophy: open, collaborative, free, open source. He also thinks that it’s a good learning resource as it shows entire sentences instead of just words, and simple sentences are good for beginners.

S\. is comfortable using browsers, command line, writing basic shell scripts.

S\. thinks his language abilities are limited both in English and French. "Even in French I do a lot of mistakes." The easiest way for him to contribute is to translate from French into English, because he feels like he does less mistakes in English than in French.

S\. was born and raised in France in a French environment, but he spent one year in the US with his siblings with when he was 8. He went to a US school and learnt English there. He often uses English to communicate with his siblings and consumes a lot of English content (articles, videos).

## Test

* Device: modern laptop with touchpad
* OS: Linux Manjaro
* Browser: Firefox
* Instructions: I did not give any precise instructions, to let S\. (re)discover Tatoeba.

### Creating flashcards

* S\. decided to show me how he used to create Anki flashcards.

#### Home page (as guest)

* S\. logs in using the top-right button.

#### Home page (as logged-in user)

* S\. opens vi in a terminal and goes back to Firefox.
* In a terminal, S\. types in a new vocabulary list that he is actually learning now at his job: tide, swell, wave, current, nose, tail
* S\. opens up Baidu translate in a new tab because he often uses it.
* S\. goes back to Tatoeba tab and types "tide" in the search bar and press enter.
* Before the page finished loading, he clicks on the back button and fills the "from" field to English. S\. type "ch" in the "to" field but is confused by the many results. S\. types "chinese" and look at the results, then clicks on Mandarin. S\. click on search button.

#### Search results (keywords=tide, from=*English*, to=*Mandarin Chinese*)

* No results.
* S\. types "swell" in the search bar and press Enter.

#### Search results (keywords=swell, from=*English*, to=*Mandarin Chinese*)

* No results. S\. asks me if he did something wrong or there are just not enough contributions.
* S\. types "swell" in the search bar and press Enter.

#### Search results (keywords=swell, from=*English*, to=*Mandarin Chinese*)

* S\. first sees sentences containing "waved" and "waving"
* S\. scrolls down and stops on a sentence containing "waves". "Perfect, it’s what I need".
* S\. clicks on the pound icon to open the sentence.

#### Sentence 995878

* S\. realizes there is no Pinyin on the Chinese sentence, so he goes back to the search results and opens the Chinese sentence instead.

#### Sentence 969829

* S\. selects the English translation, copies and pastes it into his text editor and appends a comma.
* S\. selects the simplified Chinese sentence, copies and pastes it into his text editor and appends a comma.
* S\. selects the Pinyin transcription, copies and pastes it into his text editor and appends a comma.
* S\. accidentally removes the CSV line he just created, and does all the copy and pasting again.
* S\. explains he’d import the CSV file inside Anki to learn the new vocabulary, using English as front and Chinese as back or the opposite.

### Exploration

* S\. goes through the "Contribute" menu entries to remember how he used Tatoeba. He mentions that he is not comfortable with "Adopt", "Discuss" and "Improve" because he is not confident with his language abilities and he fears doing something wrong as a newcomer.
* S\. clicks on "Add sentences"

#### Add new sentences

* S\. wonders if he ever used that. He decides to check if he did.
* S\. clicks on "My sentences" from the top-right user menu.

#### My sentences

* S\. scrolls his sentences. He remembers that at the beginning, he wanted to use Tatoeba as a translator. He wished that people would translate his sentences, but nobody did even after waiting for a long time. That’s why he started to use HiNative instead. S\. opens HiNative in a new tab.

#### HiNative homepage

* "You ask a question and the community answers."
* "A lot of people just ask 'how do you say this in English?'"
* "I think they have their own staff working because you get answers within a day."
* S\. clicks on the top-left logo to check if anything changed since the last time he used Tatoeba.

#### Homepage (as logged-in user)

* S\. asks me if the messages in the right column are the Wall latest messages.
* S\. clicks on the "Stats per languages" button. "There wasn’t stats before."

#### Number of sentences per language

* "I like that page."
* S\. clicks on the "Show activity timeline" page. "I’ve never seen this."

#### Activity timeline

* S\. is a bit confused. "What’s that? The number of people contributing? Usage stats? By month?"
* "In January, there have been 2000 people doing what? I don’t know what they did."
* S\. sees the word "contributions" in the URL. "Ah, contributions! It’s 2000 contributions. On the 1st of January. Then 1800 on the 2nd."
* S\. clicks on the "Discuss sentences" menu entry to give his feedback to me.

#### Comments on sentences

* It takes some time for S\. to understand the meaning of the 3 rows of a comment. "Okay, I got it."
* S\. scrolls down to find a language he understands.
* "I never did that because I seldom see errors in others’ sentences." "I’m not linguist enough."
* S\. clicks on the "Improve sentences" menu entry.

### Native checking

#### Improve sentences

* "What’s the difference between discuss and improve? Improve is about contributing a modification while discuss is about commenting, right?"
* S\. reads the "Tags you should know about" section.
* "How can I see all the sentences of a given languages needing a native check? It should be in 'Improve sentence'"
* S\. clicks on the "@needs native check" link.

#### Sentences with tag @needs native check.

* S\. scrolls down. "But can I see it directly on the sentence?"
* S\. clicks on the Chinese sentence from before from another tab.

#### Sentence 969829

* S\. is looking for tags. He scrolls down and then back up. After a few seconds, he finds the Tags block and clicks on "View all tags".

#### All existing tags

* S\. scrolls down. "So the @ tags are the ones you defined, while the others are what? Contributors’ tags?"
* S\. clicks on tag List 907. "What the heck is that?"

#### Sentences with tags List 907

* S\. is confused. "It’s a tag on a giant corpus. Is is Tom’s corpus? No. I don’t get it."
* S\. clicks on the back button.

#### All existing tags

* S\. clicks on the @needs native check tag.

#### Sentences with tag @needs native check

* S\. tries to filter by lang. S\. examines the URL. After a few seconds, he finds the language drop down on the right.
* S\. scrolls down to F but cannot find French (because it’s at the beginning of the list, as a profile language).
* S\. types the f key and press enter.

#### Sentences with tag @needs native check (filtered to French sentences)

* "So these sentences have been tagged as 'needs native speaker check', so I just need to read them, and if anything looks strange, I can..."
* "There aren’t a lot of them. How come? A lot of French contributors checking already? Or not a lot of people tagging?"
* "I feel lost. I don’t know what to do now. Ah yeah, the contribute menu."
* S\. clicks on Adopt sentences.

#### Orphan sentences in English

* S\. reads the first sentence "'I intended to have been a teacher.' That’s so wrong."
* "So what if I wanna...? I need to check the rules. I’d like to contribute but I’m not sure what I need to do."
* S\. clicks on "Improve sentences"

#### Improve sentences

* S\. reads the first section. "So I need to tag it 'needs to be changed'. But let’s read the rest too: How to help"
* S\. opens the "advanced contributor" link on a separate tab and keeps reading.
* "I’m not sure to get the difference between @check and @change." S\. reads again. "Okay, I got it."
* S\. switch to the newly-opened FAQ tab.

#### FAQ (wiki)

* S\. scrolls down and up, starts reading the first section. "It’s not what I need now."
* S\. switch to tab sentence 19068.

#### Sentence 19068

* S\. wants to tag this sentence @change.
* S\. hover the sentence buttons Adopt, Add to list, favorite, translate.
* S\. think about how to rephrase the sentence. He is not sure, and mentions he’s not comfortable fixing others’ sentences.
* S\. writes a comment "@change I intended to be a teacher"
* S\. looks up information about the word "intended to" on Google, reads two forum threads. "I’d have given up already because I’m too unsure. Well, I think I’m correct. But I’m not sure. But I’d say it like that."
* S\. submits the comment. "Maybe I have it all wrong... We’ll see."

### Translating

* I suggest adding a translation because S\. used to contribute that way. S\. clicks "Translate sentences".

#### Translate sentences

* S\. selects: Sentences in French not directly translated into English
* "Random. That’s new. I like it."
* S\. clicks Show sentences. "Or Fewest words also. For simple sentences."

#### Advanced search

* S\. ignores the first sentence because it’s too complex.
* S\. feels he can translate the second sentence.
* S\. clicks on the pound icon.

#### Sentence 1118122

* "I don’t remember how to do that."
* After one second, S\. finds the translate button "Ah, right, translate." and clicks it.
* S\. selects English in the drop down.
* S\. types a translation: She is my relative from my fathers side.
* S\. hesitates between "on" and "from" and looks up "relative from my fathers side" and "relative from my fathers side" on Google on two separate tabs.
* S\. reads a forum thread and decides "on" is better.
* S\. reads again the original sentence and his translation, scrolls down and up, and clicks Submit translation.

### Lists

* S\. clicks on "Browse by list". "I haven’t explored that yet."

#### All public lists

* "These are the different corpus."
* S\. clicks on "[Eutropius] Breviarium Historiae Romanae"

#### List [Eutropius] Breviarium Historiae Romanae

* "Okay so it’s one kind of contribution."
* S\. goes back.

#### All public lists

* S\. clicks on grammar. "Ah, anybody can create lists. I wanna know more about lists."
* S\. looks at the list sorting options, the Browse menu, the right menu, clicks on My lists.

#### My lists

* "So I can create lists."
* S\. clicks on Collaborative lists.

#### Collaborative lists

* "This is to make sentences more useful, by categories. For example, all sentences of beginner Chinese, or all sentences about space, right? That’s how I understand it. Trying to group sentences."
* "Anyway I’m not interested in this, but maybe I’ll use it."

### Wall

* I asked about the Wall. "Yeah, I read it when I was bored (laugh). I like to observe the communities, see what happens, who speaks, trying to understand the dynamics of the community. But it takes time, so I only did it when I had time. I also does it on Github, I look at the issues, even though I’m not a developer, because I like to see who makes things progress, what problems they face..."

### Finding the CSV export

* I asked S\. if he could find the CSV export again.
* S\. explores the top menu to no avail.
* S\. decides to search on Github. He searches "tatoeba" on Github.

#### Github

* "I’m not even sure you’re still on Github since now it’s Microsoft, maybe you switched to Gitlab?"
* S\. finds our Github repository. "Is that you? Maybe not. Yes, it’s you."
* S\. is unsure where to look. He looks up "csv dump" in our repository. "No, I’m not going to find it this way. Maybe on the wiki?"
* S\. goes on the Github wiki. "I need to Google it."
* S\. looks up "download taoeba data" on Google and clicks on the first result.

#### Downloads

* "That’s it. So this needs to be looked up from the Internet."

version at: 04/02/2020, 11:52

# User experience test

Test performed by gillux on the 31/01/2020.

## User background

* Nationality: French
* Languages: French (native), English (advanced)
* Job: helper at a surf hostel
* Age: 28

S\. joined Tatoeba in 2012 and contributed for a short time. He hasn’t been using it much since. In 2012, he used Tatoeba to learn basic Chinese vocabulary. His main use case was the creation of Anki flashcards. S\. was manually copy and pasting sentences into a text editor to create CSV data for Anki. If S\. was about to use Tatoeba again now, he’d use it exactly the same way.

At some point, S\. downloaded the CSV exports, tried play around with it to avoid copy and pasting things. However, as S\. is not a developer, he didn’t manage to complete the script he wanted to automate his process.

S\. is much interested in collaborative projects like Wikipedia. He values Tatoeba because of its philosophy: open, collaborative, free, open source. He also thinks that it’s a good learning resource as it shows entire sentences instead of just words, and simple sentences are good for beginners.

S\. is comfortable using browsers, command line, writing basic shell scripts.

S\. thinks his language abilities are limited both in English and French. "Even in French I do a lot of mistakes." The easiest way for him to contribute is to translate from French into English, because he feels like he does less mistakes in English than in French.

S\. was born and raised in France in a French environment, but he spent one year in the US with his siblings with when he was 8. He went to a US school and learnt English there. He often uses English to communicate with his siblings and consumes a lot of English content (articles, videos).

## Test

* Device: modern laptop with touchpad
* OS: Linux Manjaro
* Browser: Firefox
* Instructions: I did not give any precise instructions, to let S\. (re)discover Tatoeba.

### Creating flashcards

* S\. decided to show me how he used to create Anki flashcards.

#### Home page (as guest)

* S\. logs in using the top-right button.

#### Home page (as logged-in user)

* S\. opens vi in a terminal and goes back to Firefox.
* In a terminal, S\. types in a new vocabulary list that he is actually learning now at his job: tide, swell, wave, current, nose, tail
* S\. opens up Baidu translate in a new tab because he often uses it.
* S\. goes back to Tatoeba tab and types "tide" in the search bar and press enter.
* Before the page finished loading, he clicks on the back button and fills the "from" field to English. S\. type "ch" in the "to" field but is confused by the many results. S\. types "chinese" and look at the results, then clicks on Mandarin. S\. click on search button.

#### Search results (keywords=tide, from=*English*, to=*Mandarin Chinese*)

* No results.
* S\. types "swell" in the search bar and press Enter.

#### Search results (keywords=swell, from=*English*, to=*Mandarin Chinese*)

* No results. S\. asks me if he did something wrong or there are just not enough contributions.
* S\. types "swell" in the search bar and press Enter.

#### Search results (keywords=swell, from=*English*, to=*Mandarin Chinese*)

* S\. first sees sentences containing "waved" and "waving"
* S\. scrolls down and stops on a sentence containing "waves". "Perfect, it’s what I need".
* S\. clicks on the pound icon to open the sentence.

#### Sentence 995878

* S\. realizes there is no Pinyin on the Chinese sentence, so he goes back to the search results and opens the Chinese sentence instead.

#### Sentence 969829

* S\. selects the English translation, copies and pastes it into his text editor and appends a comma.
* S\. selects the simplified Chinese sentence, copies and pastes it into his text editor and appends a comma.
* S\. selects the Pinyin transcription, copies and pastes it into his text editor and appends a comma.
* S\. accidentally removes the CSV line he just created, and does all the copy and pasting again.
* S\. explains he’d import the CSV file inside Anki to learn the new vocabulary, using English as front and Chinese as back or the opposite.

### Exploration

* S\. goes through the "Contribute" menu entries to remember how he used Tatoeba. He mentions that he is not comfortable with "Adopt", "Discuss" and "Improve" because he is not confident with his language abilities and he fears doing something wrong as a newcomer.
* S\. clicks on "Add sentences"

#### Add new sentences

* S\. wonders if he ever used that. He decides to check if he did.
* S\. clicks on "My sentences" from the top-right user menu.

#### My sentences

* S\. scrolls his sentences. He remembers that at the beginning, he wanted to use Tatoeba as a translator. He wished that people would translate his sentences, but nobody did even after waiting for a long time. That’s why he started to use HiNative instead. S\. opens HiNative in a new tab.

#### HiNative homepage

* "You ask a question and the community answers."
* "A lot of people just ask 'how do you say this in English?'"
* "I think they have their own staff working because you get answers within a day."
* S\. clicks on the top-left logo to check if anything changed since the last time he used Tatoeba.

#### Homepage (as logged-in user)

* S\. asks me if the messages in the right column are the Wall latest messages.
* S\. clicks on the "Stats per languages" button. "There wasn’t stats before."

#### Number of sentences per language

* "I like that page."
* S\. clicks on the "Show activity timeline" page. "I’ve never seen this."

#### Activity timeline

* S\. is a bit confused. "What’s that? The number of people contributing? Usage stats? By month?"
* "In January, there have been 2000 people doing what? I don’t know what they did."
* S\. sees the word "contributions" in the URL. "Ah, contributions! It’s 2000 contributions. On the 1st of January. Then 1800 on the 2nd."
* S\. clicks on the "Discuss sentences" menu entry to give his feedback to me.

#### Comments on sentences

* It takes some time for S\. to understand the meaning of the 3 rows of a comment. "Okay, I got it."
* S\. scrolls down to find a language he understands.
* "I never did that because I seldom see errors in others’ sentences." "I’m not linguist enough."
* S\. clicks on the "Improve sentences" menu entry.

### Native checking

#### Improve sentences

* "What’s the difference between discuss and improve? Improve is about contributing a modification while discuss is about commenting, right?"
* S\. reads the "Tags you should know about" section.
* "How can I see all the sentences of a given languages needing a native check? It should be in 'Improve sentence'"
* S\. clicks on the "@needs native check" link.

#### Sentences with tag @needs native check.

* S\. scrolls down. "But can I see it directly on the sentence?"
* S\. clicks on the Chinese sentence from before from another tab.

#### Sentence 969829

* S\. is looking for tags. He scrolls down and then back up. After a few seconds, he finds the Tags block and clicks on "View all tags".

#### All existing tags

* S\. scrolls down. "So the @ tags are the ones you defined, while the others are what? Contributors’ tags?"
* S\. clicks on tag List 907. "What the heck is that?"

#### Sentences with tags List 907

* S\. is confused. "It’s a tag on a giant corpus. Is is Tom’s corpus? No. I don’t get it."
* S\. clicks on the back button.

#### All existing tags

* S\. clicks on the @needs native check tag.

#### Sentences with tag @needs native check

* S\. tries to filter by lang. S\. examines the URL. After a few seconds, he finds the language drop down on the right.
* S\. scrolls down to F but cannot find French (because it’s at the beginning of the list, as a profile language).
* S\. types the f key and press enter.

#### Sentences with tag @needs native check (filtered to French sentences)

* "So these sentences have been tagged as 'needs native speaker check', so I just need to read them, and if anything looks strange, I can..."
* "There aren’t a lot of them. How come? A lot of French contributors checking already? Or not a lot of people tagging?"
* "I feel lost. I don’t know what to do now. Ah yeah, the contribute menu."
* S\. clicks on Adopt sentences.

#### Orphan sentences in English

* S\. reads the first sentence "'I intended to have been a teacher.' That’s so wrong."
* "So what if I wanna...? I need to check the rules. I’d like to contribute but I’m not sure what I need to do."
* S\. clicks on "Improve sentences"

#### Improve sentences

* S\. reads the first section. "So I need to tag it 'needs to be changed'. But let’s read the rest too: How to help"
* S\. opens the "advanced contributor" link on a separate tab and keeps reading.
* "I’m not sure to get the difference between @check and @change." S\. reads again. "Okay, I got it."
* S\. switch to the newly-opened FAQ tab.

#### FAQ (wiki)

* S\. scrolls down and up, starts reading the first section. "It’s not what I need now."
* S\. switch to tab sentence 19068.

#### Sentence 19068

* S\. wants to tag this sentence @change.
* S\. hover the sentence buttons Adopt, Add to list, favorite, translate.
* S\. think about how to rephrase the sentence. He is not sure, and mentions he’s not comfortable fixing others’ sentences.
* S\. writes a comment "@change I intended to be a teacher"
* S\. looks up information about the word "intended to" on Google, reads two forum threads. "I’d have given up already because I’m too unsure. Well, I think I’m correct. But I’m not sure. But I’d say it like that."
* S\. submits the comment. "Maybe I have it all wrong... We’ll see."

### Translating

* I suggest adding a translation because S\. used to contribute that way. S\. clicks "Translate sentences".

#### Translate sentences

* S\. selects: Sentences in French not directly translated into English
* "Random. That’s new. I like it."
* S\. clicks Show sentences. "Or Fewest words also. For simple sentences."

#### Advanced search

* S\. ignores the first sentence because it’s too complex.
* S\. feels he can translate the second sentence.
* S\. clicks on the pound icon.

#### Sentence 1118122

* "I don’t remember how to do that."
* After one second, S\. finds the translate button "Ah, right, translate." and clicks it.
* S\. selects English in the drop down.
* S\. types a translation: She is my relative from my fathers side.
* S\. hesitates between "on" and "from" and looks up "relative from my fathers side" and "relative from my fathers side" on Google on two separate tabs.
* S\. reads a forum thread and decides "on" is better.
* S\. reads again the original sentence and his translation, scrolls down and up, and clicks Submit translation.

### Lists

* S\. clicks on "Browse by list". "I haven’t explored that yet."

#### All public lists

* "These are the different corpus."
* S\. clicks on "[Eutropius] Breviarium Historiae Romanae"

#### List [Eutropius] Breviarium Historiae Romanae

* "Okay so it’s one kind of contribution."
* S\. goes back.

#### All public lists

* S\. clicks on grammar. "Ah, anybody can create lists. I wanna know more about lists."
* S\. looks at the list sorting options, the Browse menu, the right menu, clicks on My lists.

#### My lists

* "So I can create lists."
* S\. clicks on Collaborative lists.

#### Collaborative lists

* "This is to make sentences more useful, by categories. For example, all sentences of beginner Chinese, or all sentences about space, right? That’s how I understand it. Trying to group sentences."
* "Anyway I’m not interested in this, but maybe I’ll use it."

### Wall

* I asked about the Wall. "Yeah, I read it when I was bored (laugh). I like to observe the communities, see what happens, who speaks, trying to understand the dynamics of the community. But it takes time, so I only did it when I had time. I also does it on Github, I look at the issues, even though I’m not a developer, because I like to see who makes things progress, what problems they face..."

### Finding the CSV export

* I asked S\. if he could find the CSV export again.
* S\. explores the top menu to no avail.
* S\. decides to search on Github. He searches "tatoeba" on Github.

#### Github

* "I’m not even sure you’re still on Github since now it’s Microsoft, maybe you switched to Gitlab?"
* S\. finds our Github repository. "Is that you? Maybe not. Yes, it’s you."
* S\. is unsure where to look. He looks up "csv dump" in our repository. "No, I’m not going to find it this way. Maybe on the wiki?"
* S\. goes on the Github wiki. "I need to Google it."
* S\. looks up "download taoeba data" on Google and clicks on the first result.

#### Downloads

* "That’s it. So this needs to be looked up from the Internet."

## Identified problems

* Tatoeba doesn’t match the user needs:
  * S\. wished that people would translate his sentences, but nobody did even after waiting for a long time.
  * S\. wants to create Anki decks with English/Chinese/Pinyin but he needs to copy and paste everything by hand.

* Usability problems:
  * Mandarin Chinese requires extra effort to be found in the drop down. There are many language names that include "Chinese" and Mandarin happens to be the last entry, so one needs to either scroll the dropdown or additionally type "mandarin".
  * S\. is looking at an English sentence and wants to see the Pinyin on the translated sentence, but it’s not displayed. However it makes sense to display it since he is doing an English→Chinese search.
  * A search showing "No results" doesn’t tell if it’s because of the selected criteria (like "translated into") or because the keywords are not matching anything.
  * The "Filter by language" block is hard to find, when located on the top-right of the page, like on the "comments on sentence" or "tag @needs native check" pages.
  * The "Improve sentences" page lists instructions that are too complex.
  * On a sentence page, the "Tags" block, when empty, shows the "Show all tags" link, which makes S\. think he needs to click on it to see the tags of that particular sentence.
  * Tag names, like "List 907", are not self-explanatory enough.
  * The HTML language drop down has profile languages moved up, so these cannot be found when going through the whole list by alphabetical order.
  * On the "Improve sentences" page, the hyperlink of "advanced contributor" brings to the head of the FAQ page instead of the [appropriate page](https://en.wiki.tatoeba.org/articles/show/advanced-contributors).

* Feeling problems:
  * S\. is not confident with his language abilities and he fears doing something wrong as a newcomer.

* Onboarding problems:
  * The activity timeline page is hard to understand.
  * The 3-lines blocks on the page "comments on sentences" are hard to understand.

Note

The lines in green are the lines that have been added in the new version. The lines in red are those that have been removed.

Actions

• History
• Revert to version made the 31/01/2020, 18:13
• Go back to article's page