Version at: 09/02/2020, 11:16

# User experience test

Test performed by gillux on the 08/02/2020.

## User background

* Nationality: Japanese
* Languages: Japanese (native), Chinese Mandarin (advanced), English (very basic)
* Job: Japanese teacher and translator
* Age: 28

## Test

* Device: modern laptop with touchpad
* OS: Windows 10
* Browser: Chrome
* Instructions: I did not give any precise instructions, to let M\. discover Tatoeba.

I already quickly showed M\. to Tatoeba in the past.

### Exploration

#### Homepage as guest

* M\. looks at the Random sentence block but she cannot understand any of the sentences. She thinks is a pity that that sentence comes at the top since she is Japanese and the website is set in Japanese. It’s not interesting for her. She feels like "what is that actually for?" ("結局何これ"）
* M\. is disappointed that the content mixes Japanese and English. She feels like while the first introduction green block is in Japanese, the rest is basically English (mixed with Japanese).
* M\. clicks on the Browse by language link from the top menu. "Let’s have a look" (ちょっと見てみよう)

#### Japanese sentences (without translations, page 1 and 2)

* M\. reads the sentences and smiles at a funny one.
* "But these are only Japanese." M\. sees the drop downs on the right and sets "Show translations in" to "All languages".

#### Japanese sentences (with translations)

* M\. notices that there are two very similar sentences 日誌をつける。and 日記をつける。 added by the same member, likely consecutively. "Can’t I see such sentences grouped together?"
* M\. enters a keyword in the top bar and sets to=Japanese.

#### Search (keywords=つける, from=Japanese, To=Any language) page 1 and 2

* After looking at the results, M\. understands that the search engine doesn’t understand grammar because it shows the verb つける but also 見つける which is a totally different word. "It makes it difficult to search."
* "If I only want the verb つける, I need to search differently, like をつける"

#### Search (keywords=をつける, from=Japanese, To=Any language) page 1 and 2

* (pointing out the number of results getting lower) "This way I can narrow down the results."
* M\. looks at the sentences
* M\. looks at the additional search filters on the right. She changes "Show translations in:" to "None".
* M\. looks at the other filters. "There are many ways to search."
* M\. goes through all the filters.
* M\. thinks that a filter like "by username" doesn’t need to be on the top because it’s probably only used by very active members (I had mentioned to her that our community include such members) while first-time users would certainly don’t understand it and be confused.
* "What does "unapproved" (非承認) mean? That it hasn’t been checked? (チェックされてない）. It’s probably that."
* M\. clicks on the "Belongs to list" drop down. It’s very large and very long "What the hell is that? That’s very strange." (謎のものが出てきてる)
* M\. eventually clicks the submit button.

#### Search (keywords=をつける, from=Japanese, To=Any language) without showing translations

* M\. explains that she wasn’t able to visually compare the sentences, so she I disabled the translations. She suggests having that feature as a per-sentence collapse/expand button instead. The translation would be hidden by default if I’m only searching for Japanese sentences.
* M\. clicks on Browse by list from the top menu.

#### All public lists

* "What the hell is this?" ([どういうこと？](https://tatoeba.org/sentences/show/3553284))
* M\. clicks on the first list

#### List [Rosinus] Antiquitatum Romanarum Corpus Absolutissimum

* M\. clicks on the back browser button after a few seconds.

#### All public lists

* M\. looks at the lists. "Let’s ignore this."
* M\. clicks on the "Browse by tag" from the top menu.

#### All tags

* M\. clicks on the "Browse audio" from the top menu.

#### Sentences with audio

* The first page only contains English sentences. "The default language is probably English"
* M\. looks at the English (UK) flag and remembers that in the past I mentioned to her the flag problem. "It makes people wrongly assume it’s British English. I would misunderstand that. Maybe. I didn’t really think about it at first, but if I were to study English, I’d certainly want to know more about the differences among English dialects. For example the way Chinese in spoken in Taiwan is very different from mainland China, to the extend that it’s sometimes hard for Taiwanese to understand mainlanders. I wonder if it’s possible to indicate such differences using the flag."
* M\. plays an English recording. "Aren’t people who listen to recordings interested in this too? Where is the speaker from? I don’t really know about English, but it’s probably true for Japanese too."
* M\. clicks on the Japanese number.

#### Sentences in Japanese with audio

* M\. plays the first sentence: ここに座ってもいいですか？ "She’s very upset, isn’t she? It doesn’t feel like a question."
* M\. mentions that there are sentences in both polite and informal forms, but learners of Japanese (especially learning on their own) might not be able to tell wether a sentence is formal, informal, written, spoken… It may be easier to use if we could distinguish sentences at a glance.
* M\. plays a few other sentences.
* M\. clicks on "Add sentences" from the top menu.

#### Login

* "I need to login, I see. Same for 'Add translations', right?"
* "What is 'Adopt sentences'?" M\. clicks.

#### Orphan sentences in Japanese

* M\. sees and reads the "About adoption" block on the right.
* M\. feels confused after reading the first sentences, but ultimately understands after she finishes reading the whole text. "I see, I got it."
* "So if I adopt someone else’s sentence, I can correct it, right?"
* "Are there lots of people wanting to do that? Fixing awkward sentences. I wonder."
* M\. clicks "Improve sentences".

#### Improve sentences

* Now that M\. knows the meaning of adopting, she wonders what’s the difference between "Adopt sentences" and "Improve sentences". "Isn’t it the same thing?"
* M\. looks at the first section. "So this 'improve sentences' is actually… tagging? It says 'improve sentences', but it's only about tagging. Shouldn’t you guys group adopt and improve into a single thing?"
* M\. clicks on Discuss sentences from the top menu.

#### Discuss sentences (例文について議論する)

* M\. sees a comment on a Japanese sentence that looks like answering a question. She wonders if people also use this feature to do some real debating too. (The Japanese word 議論 actually means to debate, to argue with the goal of finding consensus or an answer.)
* M\. scrolls down. "I don’t understand this language."
* While looking at the sentence "このチケット、欲しい人に差し上げて。", M\. says it’s strange. M\. clicks on the name of the sentence owner. "Who is this?"

#### Profile page of small_snow

* "This person is probably a native speaker of Japanese."
* M\. clicks on the back browser button.

#### Discuss sentences

* M\. clicks on "Show activity timeline" from the top menu (this entry is not translated in Japanese).

#### Show activity timeline

* After a few seconds, M\. clicks on Wall from the top menu.
* I prompt M\. to explain why she ignored the page. M\. checks the page again. "Number of views? No. Number of added sentences? I don’t know. I have no interest in this. I wonder if it’s worth putting that on the top menu."

#### Wall (掲示板)

* "Someone’s writing about the updates."
* "They say 'thank you' in a lot of languages. :-)"
* "This user posts a lot. I saw his posts elsewhere."
* "A lot of people are talking there…"
* M\. points out that even though this page is called bulletin board (掲示板), it looks more about insiders who work on the project talking to each other rather than simple users socializing. It’s interesting/funny to see how the project is running, but at the same time M\. feels a bit confused/surprised to suddenly see messages about technical stuff like Aiji’s weekly recap. But M\. thinks it’s okay since it’s possible to filter out by topic.
* Then, M\. realize it’s not possible to filter by topic. "There are 5680 threads, but if I want to find a particular thread that I saw in the past, I need to go through all of them page by page? That’d be difficult."
* \M. clicks on "Languages of members"

#### Languages of members

* "There are quite a lot of people of European languages. Russian Portuguese are surprisingly high."
* "It’s interesting, it looks like a population distribution graph."
* M\. clicks on "Japanese"

#### Members: Japanese

* "So many people out there. Wow, one of them is called "Japanese" (nihonjin)!"
* The page shows only members with five stars. M\. wonders if a lot of stars means that the user contributed a lot of sentences.
* M\. goes to page 2, 3 and still sees only members with five stars.
* M\. goes to the last page (77) and sees only users without stars. "Ah, but some of them do not have stars."
* M\. clicks on the last one Thomas13.

#### Profile of Thomas13

* The profile shows Portuguese with five stars, and some other languages without stars.
* "Ah, so it means which languages... No, it can’t be that. Ah, it’s about which language the user is a native speaker of, about the level? I wonder."
* M\. clicks on the back browser button twice.

#### Members: Japanese (page 3)

* M\. browses to pages 7, 77, 71 (shows empty star users) "Why don’t they have stars?", 68 (shows one star users), 71, and clicks on user Njord.

#### User profile Njord

* The profile shows languages with various number of stars.
* M\. concludes the number of stars represents the level.
* "So zero stars mean the user is studying the language? Interesting"
* M\. clicks on the back browser button twice.

#### Members: Japanese

* "But I guess some members did nothing but register."
* M\. clicks on Native speakers from the top menu (this entry is not translated in Japanese)

#### Native speakers

* M\. smiles at the Legend and goes through the list of languages.
* "There’s even Literary Chinese."
* "There are languages that only language geeks know" (めちゃめちゃマニアックな言葉とかもあるんだね)
* "There’s also Hakka."
* "Wow, Ainu is also supported?"
* (Looking at the number of the first column) "zero"
* M\. considers herself done with exploring the site. She is more interested in translating than adding new sentences.
* M\. clicks on Login (ログイン), the login pane shows, "It’s not that", and she clicks on Register (登録).

#### Register

* M\. fills all the fields without a hitch.

#### Home page as logged-in user

* M\. reads the welcome message. "I’m being welcomed. :-)"
* M\. clicks on the profile link.

### Contributing

#### My profile

* "I’m a normal user (一般ユーザ, translation for "contributor")
* M\. clicks on the top-right Edit button

#### Edit profile

* M\. enters her name, country and birthday date. "Will something happen on my birthday? They will say happy birthday to me?"
* "I don’t need to enter things like website."
* M\. is confused: "But where is the language setting they talked about? It’s not there, right?"
* M\. clicks on the Save button "I’ll save these settings for now."

#### My profile

* M\. looks at her page.
* "Ah, there it is." M\. clicks the down-right Add a language button

#### Add a language

* M\. scrolls the language list for 50 seconds before she finds what she wants. She looks at the kanji-named languages, then katakana-named languages, then Latin-script-named languages until she finds Mandarin Chinese (this language name hasn’t been translated into Japanese).
* M\. looks at the Level field. "I’m not sure. Let’s set it in-between."
* M\. clicks the submit button.

#### My profile

* "Is that it? The registration is finished already? That was so quick."
* M\. decides to explore the new personal menu that appeared since she registered.
* M\. clicks on "My sentences" from the user menu.

#### My sentences

* M\. clicks on "My vocabulary" from the user menu.

#### My vocabulary

* M\. clicks on "My collection" from the user menu (this one is not translated into Japanese).

#### My collection

* "I don’t have any lists of my own yet, so obviously nothing shows up."
* "It probably means I can do collections."
* M\. clicks on Browse by language from the top menu

#### Sentences in Japanese

* M\. clicks on the "Sentences in:" drop down. She looks for Mandarin Chinese, but cannot find it in the list because it is located on the top of the list because it’s a profile language. M\. goes through the list for about 42 seconds before she finds it.

#### Sentences in Mandarin Chinese

* M\. looks at the sentences. She finds "我好孤单。" funny.
* "What’s this sharp symbol?" M\. clicks it.

#### Sentence 8517170

* "Ah, the details can be seen there."
* M\. hovers the sentence buttons and reads the tooltips: "Add translation", "Add to favorites" and "Add to list".
* M\. wonders what is "Add to list" and clicks it.
* M\. clicks the list dropdown. "What the hell is that? That looks scary, let’s get out of here."
* M\. clicks the Add to favorites button. "Let’s add it to my favorites."
* M\. clicks on the back browser button.

#### Sentences in Mandarin Chinese

* "Since I registered, I thought I’ll start by adding some translations, but I couldn’t figure out how to find sentences that I’d like to translate, so I assumed that I need to go through all of the Chinese sentences, and now I feel like I’ve lost my motivation.
* "So if I want to translate these sentences in Japanese, I first need to check if there is already a Japanese translation. Isn’t there a way to search by excluding Japanese? The filter 'Show translations in:' allows to narrow down by language, but can’t I exclude Japanese? So I need to go through every sentence to see whether there is a Japanese translation or not."
* M\. clicks on the Add translation button of a Chinese sentence. "Let’s try to translate this one."
* M\. enters a translation.
* M\. selects "Japanese" as language.
* M\. clicks submit. The translation is added.
* M\. wants to add an audio recording and clicks the speaker button.
* "They kinldy explains how to add a recording. :-)"

#### Tatoeba wiki page "Contribute audio for Tatoeba"

* "What, it’s in English? Well, I don’t wanna read this. Let’s forget about it. Goodbye." M\. clicks the back browser button.

#### Sentences in Mandarin Chinese

* I ask M\. what she wanted to do. M\. decides to read the wiki page in Japanese using the Google translate button in her browser.

#### Tatoeba wiki page "Contribute audio for Tatoeba" (Google-translated version)

* M\. reads the page. "I understand what the page says, but eventually I don’t understand how to contribute audio. The only thing I understand is that audio is contributed by people."
* M\. goes back to the sentences list tab.

#### Sentences in Mandarin Chinese

* M\. clicks on the arrow icon of her translation

#### Sentence 8532850

* "So I can always edit sentences that have been added by me."
* M\. clicks the back browser button

#### Sentences in Mandarin Chinese

* M\. adds another translation. "Let’s add like five translations."
* M\. wonders if something will happen after she adds more translations, but she guesses probably not.

## Debrief

* I asked M\. about how she feels about adding translations.

My impressions about adding these few translations is that I don’t feel anything. Using this website, it feels like nothing comes back even though I did something. There is no sense of accomplishment for contributors. After adding many sentences, I guess other people from the community will give me some feedback, like on the bulletin board. But it’s simply boring.

The most important thing for language learners is motivation, but this website doesn’t maintain motivation. And how do get motivation? By getting a feedback of some sort. For example, let’s say I just registered. Then it says 'Thank you for registering! Alright, now let’s write a new sentence!'. It’s not really guidance, but if there were something like that, I’d easily understand what to do next, so I can directly move on to the next thing. However after I registered, nothing happened, so I didn’t know what to do.

This website is really light (あっさりしてる), as both in pejorative and meliorative senses of the word.

* I asked M\. if she thinks Tatoeba could be of any use to her in as a teacher of Japanese.

If audio recordings were to become the main part of the project, if the number of recordings were to grow, I’d introduce Tatoeba to my students as a tool they can use when they study alone since short sentences with audio are easy to study.

Also, sentences are contributed by individuals, so quite a lot of sentences are not like what you can find on textbooks, and that’s interesting. For example, "さもなければ、死ぬよ。" I wonder when one would use that (laugh).

People who study Japanese maybe just study for studying. A lot of them are in contact with the language only during the time they study it. It’s a pity because they are rarely in contact with anything else than textbook-like contents, as opposed to what is actually used for example in newspapers, magazines or novels. They don’t have much opportunities to see actual living language. While in places like Tatoeba, you have sentences like 彼女が飲むのが好きだ。(She likes to drink.). This is something you’d never see in a textbook. It means drinking alcohol. In a textbook, you’d only see instead 彼女がお酒を飲むのが好きだ。(She likes to drink alcohol.) And that’s because it was written by a native speaker. I like that.

It depends on the way Tatoeba is used. There is an app I can introduce to students saying that they can learn new words by studying a little bit everyday. But in the case of Tatoeba, it would be difficult for me to introduce it to students. I wouldn’t know how to explain. There are other example sentence websites aimed at language learners, but I can’t really tell what’s the difference between them and Tatoeba. Well, if anything, I can say that Tatoeba includes native speaker sentences, but then again not all of them are. For example, that one, 彼は英語を大声で話す, I translated it into Chinese, but I doubt anyone would use a sentence like that in real life.

I think Tatoeba would be easier for me to use if it made it clearer for what purpose students actually use Tatoeba. For example, let’s say I wanna study spoken/colloquial language, so it would be useful if I could search only into colloquial sentences. Or let’s say that I’m reading a novel, and there is a sentence I don’t understand, and I wanna see how to use a particular word. It would be useful if I could search only into sentences of written language. Right now I can’t do that. If the search results include a translation in my mother tongue, I can guess whether the original sentence is of written or spoken language, but honestly I don’t know if I can trust the translation on a website like this where many people contribute. Eventually it’s hard to draw a conclusion from the search results. I can’t really know whether a given word should be used in this or that way.

* I asked M\. to try to use the Translate sentences page from the top menu since she complained about not being able to exclude sentences already having a Japanese translation. She used the page but she completely ignored the criteria "Not directly translated into:" (直接訳があれば除外：). It turned out she thought it was about filtering literal translations, because the Japanese for "literal translation" (直訳) and "direct translation" (直接訳) are very similar. I explained her the concept of direct versus indirect translations and showed her the advanced search criterion "Link: direct/indirect/any". M\. said that she did understand the link criterion because the two words are displayed next to each other (直接訳・間接訳), but on the Translate sentences page, 直接訳 is used alone, so it’s confusing.

## Identified problems

* Tatoeba doesn’t match the user needs:

* Usability problems:
  * Even though M\. used the top menu "Add translations" link as guest and saw a login page, once registered, she didn’t click that link to translate sentences. Instead, she used the "browse sentences" link to browse Chinese sentences and complained that you have to manually check if it’s already translated.
  * At some point during the exploration phase, using the new design, M\. clicks on a sentence text several times thinking it would do something but nothing happens.

* Feeling problems:

* Onboarding problems:

version at: 09/02/2020, 11:42

# User experience test

Test performed by gillux on the 08/02/2020.

## User background

* Nationality: Japanese
* Languages: Japanese (native), Chinese Mandarin (advanced), English (very basic)
* Job: Japanese teacher and translator
* Age: 28

## Test

* Device: modern laptop with touchpad
* OS: Windows 10
* Browser: Chrome
* Instructions: I did not give any precise instructions, to let M\. discover Tatoeba.

I already quickly showed M\. to Tatoeba in the past.

### Exploration

#### Homepage as guest

* M\. looks at the Random sentence block but she cannot understand any of the sentences. She thinks is a pity that that sentence comes at the top since she is Japanese and the website is set in Japanese. It’s not interesting for her. She feels like "what is that actually for?" ("結局何これ"）
* M\. is disappointed that the content mixes Japanese and English. She feels like while the first introduction green block is in Japanese, the rest is basically English (mixed with Japanese).
* M\. clicks on the Browse by language link from the top menu. "Let’s have a look" (ちょっと見てみよう)

#### Japanese sentences (without translations, page 1 and 2)

* M\. reads the sentences and smiles at a funny one.
* "But these are only Japanese." M\. sees the drop downs on the right and sets "Show translations in" to "All languages".

#### Japanese sentences (with translations)

* M\. notices that there are two very similar sentences 日誌をつける。and 日記をつける。 added by the same member, likely consecutively. "Can’t I see such sentences grouped together?"
* M\. enters a keyword in the top bar and sets to=Japanese.

#### Search (keywords=つける, from=Japanese, To=Any language) page 1 and 2

* After looking at the results, M\. understands that the search engine doesn’t understand grammar because it shows the verb つける but also 見つける which is a totally different word. "It makes it difficult to search."
* "If I only want the verb つける, I need to search differently, like をつける"

#### Search (keywords=をつける, from=Japanese, To=Any language) page 1 and 2

* (pointing out the number of results getting lower) "This way I can narrow down the results."
* M\. looks at the sentences
* M\. looks at the additional search filters on the right. She changes "Show translations in:" to "None".
* M\. looks at the other filters. "There are many ways to search."
* M\. goes through all the filters.
* M\. thinks that a filter like "by username" doesn’t need to be on the top because it’s probably only used by very active members (I had mentioned to her that our community include such members) while first-time users would certainly don’t understand it and be confused.
* "What does "unapproved" (非承認) mean? That it hasn’t been checked? (チェックされてない）. It’s probably that."
* M\. clicks on the "Belongs to list" drop down. It’s very large and very long "What the hell is that? That’s very strange." (謎のものが出てきてる)
* M\. eventually clicks the submit button.

#### Search (keywords=をつける, from=Japanese, To=Any language) without showing translations

* M\. explains that she wasn’t able to visually compare the sentences, so she I disabled the translations. She suggests having that feature as a per-sentence collapse/expand button instead. The translation would be hidden by default if I’m only searching for Japanese sentences.
* M\. clicks on Browse by list from the top menu.

#### All public lists

* "What the hell is this?" ([どういうこと？](https://tatoeba.org/sentences/show/3553284))
* M\. clicks on the first list

#### List [Rosinus] Antiquitatum Romanarum Corpus Absolutissimum

* M\. clicks on the back browser button after a few seconds.

#### All public lists

* M\. looks at the lists. "Let’s ignore this."
* M\. clicks on the "Browse by tag" from the top menu.

#### All tags

* M\. clicks on the "Browse audio" from the top menu.

#### Sentences with audio

* The first page only contains English sentences. "The default language is probably English"
* M\. looks at the English (UK) flag and remembers that in the past I mentioned to her the flag problem. "It makes people wrongly assume it’s British English. I would misunderstand that. Maybe. I didn’t really think about it at first, but if I were to study English, I’d certainly want to know more about the differences among English dialects. For example the way Chinese in spoken in Taiwan is very different from mainland China, to the extend that it’s sometimes hard for Taiwanese to understand mainlanders. I wonder if it’s possible to indicate such differences using the flag."
* M\. plays an English recording. "Aren’t people who listen to recordings interested in this too? Where is the speaker from? I don’t really know about English, but it’s probably true for Japanese too."
* M\. clicks on the Japanese number.

#### Sentences in Japanese with audio

* M\. plays the first sentence: ここに座ってもいいですか？ "She’s very upset, isn’t she? It doesn’t feel like a question."
* M\. mentions that there are sentences in both polite and informal forms, but learners of Japanese (especially learning on their own) might not be able to tell wether a sentence is formal, informal, written, spoken… It may be easier to use if we could distinguish sentences at a glance.
* M\. plays a few other sentences.
* M\. clicks on "Add sentences" from the top menu.

#### Login

* "I need to login, I see. Same for 'Add translations', right?"
* "What is 'Adopt sentences'?" M\. clicks.

#### Orphan sentences in Japanese

* M\. sees and reads the "About adoption" block on the right.
* M\. feels confused after reading the first sentences, but ultimately understands after she finishes reading the whole text. "I see, I got it."
* "So if I adopt someone else’s sentence, I can correct it, right?"
* "Are there lots of people wanting to do that? Fixing awkward sentences. I wonder."
* M\. clicks "Improve sentences".

#### Improve sentences

* Now that M\. knows the meaning of adopting, she wonders what’s the difference between "Adopt sentences" and "Improve sentences". "Isn’t it the same thing?"
* M\. looks at the first section. "So this 'improve sentences' is actually… tagging? It says 'improve sentences', but it's only about tagging. Shouldn’t you guys group adopt and improve into a single thing?"
* M\. clicks on Discuss sentences from the top menu.

#### Discuss sentences (例文について議論する)

* M\. sees a comment on a Japanese sentence that looks like answering a question. She wonders if people also use this feature to do some real debating too. (The Japanese word 議論 actually means to debate, to argue with the goal of finding consensus or an answer.)
* M\. scrolls down. "I don’t understand this language."
* While looking at the sentence "このチケット、欲しい人に差し上げて。", M\. says it’s strange. M\. clicks on the name of the sentence owner. "Who is this?"

#### Profile page of small_snow

* "This person is probably a native speaker of Japanese."
* M\. clicks on the back browser button.

#### Discuss sentences

* M\. clicks on "Show activity timeline" from the top menu (this entry is not translated in Japanese).

#### Show activity timeline

* After a few seconds, M\. clicks on Wall from the top menu.
* I prompt M\. to explain why she ignored the page. M\. checks the page again. "Number of views? No. Number of added sentences? I don’t know. I have no interest in this. I wonder if it’s worth putting that on the top menu."

#### Wall (掲示板)

* "Someone’s writing about the updates."
* "They say 'thank you' in a lot of languages. :-)"
* "This user posts a lot. I saw his posts elsewhere."
* "A lot of people are talking there…"
* M\. points out that even though this page is called bulletin board (掲示板), it looks more about insiders who work on the project talking to each other rather than simple users socializing. It’s interesting/funny to see how the project is running, but at the same time M\. feels a bit confused/surprised to suddenly see messages about technical stuff like Aiji’s weekly recap. But M\. thinks it’s okay since it’s possible to filter out by topic.
* Then, M\. realize it’s not possible to filter by topic. "There are 5680 threads, but if I want to find a particular thread that I saw in the past, I need to go through all of them page by page? That’d be difficult."
* \M. clicks on "Languages of members"

#### Languages of members

* "There are quite a lot of people of European languages. Russian Portuguese are surprisingly high."
* "It’s interesting, it looks like a population distribution graph."
* M\. clicks on "Japanese"

#### Members: Japanese

* "So many people out there. Wow, one of them is called "Japanese" (nihonjin)!"
* The page shows only members with five stars. M\. wonders if a lot of stars means that the user contributed a lot of sentences.
* M\. goes to page 2, 3 and still sees only members with five stars.
* M\. goes to the last page (77) and sees only users without stars. "Ah, but some of them do not have stars."
* M\. clicks on the last one Thomas13.

#### Profile of Thomas13

* The profile shows Portuguese with five stars, and some other languages without stars.
* "Ah, so it means which languages... No, it can’t be that. Ah, it’s about which language the user is a native speaker of, about the level? I wonder."
* M\. clicks on the back browser button twice.

#### Members: Japanese (page 3)

* M\. browses to pages 7, 77, 71 (shows empty star users) "Why don’t they have stars?", 68 (shows one star users), 71, and clicks on user Njord.

#### User profile Njord

* The profile shows languages with various number of stars.
* M\. concludes the number of stars represents the level.
* "So zero stars mean the user is studying the language? Interesting"
* M\. clicks on the back browser button twice.

#### Members: Japanese

* "But I guess some members did nothing but register."
* M\. clicks on Native speakers from the top menu (this entry is not translated in Japanese)

#### Native speakers

* M\. smiles at the Legend and goes through the list of languages.
* "There’s even Literary Chinese."
* "There are languages that only language geeks know" (めちゃめちゃマニアックな言葉とかもあるんだね)
* "There’s also Hakka."
* "Wow, Ainu is also supported?"
* (Looking at the number of the first column) "zero"
* M\. considers herself done with exploring the site. She is more interested in translating than adding new sentences.
* M\. clicks on Login (ログイン), the login pane shows, "It’s not that", and she clicks on Register (登録).

#### Register

* M\. fills all the fields without a hitch.

#### Home page as logged-in user

* M\. reads the welcome message. "I’m being welcomed. :-)"
* M\. clicks on the profile link.

### Contributing

#### My profile

* "I’m a normal user (一般ユーザ, translation for "contributor")
* M\. clicks on the top-right Edit button

#### Edit profile

* M\. enters her name, country and birthday date. "Will something happen on my birthday? They will say happy birthday to me?"
* "I don’t need to enter things like website."
* M\. is confused: "But where is the language setting they talked about? It’s not there, right?"
* M\. clicks on the Save button "I’ll save these settings for now."

#### My profile

* M\. looks at her page.
* "Ah, there it is." M\. clicks the down-right Add a language button

#### Add a language

* M\. scrolls the language list for 50 seconds before she finds what she wants. She looks at the kanji-named languages, then katakana-named languages, then Latin-script-named languages until she finds Mandarin Chinese (this language name hasn’t been translated into Japanese).
* M\. looks at the Level field. "I’m not sure. Let’s set it in-between."
* M\. clicks the submit button.

#### My profile

* "Is that it? The registration is finished already? That was so quick."
* M\. decides to explore the new personal menu that appeared since she registered.
* M\. clicks on "My sentences" from the user menu.

#### My sentences

* M\. clicks on "My vocabulary" from the user menu.

#### My vocabulary

* M\. clicks on "My collection" from the user menu (this one is not translated into Japanese).

#### My collection

* "I don’t have any lists of my own yet, so obviously nothing shows up."
* "It probably means I can do collections."
* M\. clicks on Browse by language from the top menu

#### Sentences in Japanese

* M\. clicks on the "Sentences in:" drop down. She looks for Mandarin Chinese, but cannot find it in the list because it is located on the top of the list because it’s a profile language. M\. goes through the list for about 42 seconds before she finds it.

#### Sentences in Mandarin Chinese

* M\. looks at the sentences. She finds "我好孤单。" funny.
* "What’s this sharp symbol?" M\. clicks it.

#### Sentence 8517170

* "Ah, the details can be seen there."
* M\. hovers the sentence buttons and reads the tooltips: "Add translation", "Add to favorites" and "Add to list".
* M\. wonders what is "Add to list" and clicks it.
* M\. clicks the list dropdown. "What the hell is that? That looks scary, let’s get out of here."
* M\. clicks the Add to favorites button. "Let’s add it to my favorites."
* M\. clicks on the back browser button.

#### Sentences in Mandarin Chinese

* "Since I registered, I thought I’ll start by adding some translations, but I couldn’t figure out how to find sentences that I’d like to translate, so I assumed that I need to go through all of the Chinese sentences, and now I feel like I’ve lost my motivation.
* "So if I want to translate these sentences in Japanese, I first need to check if there is already a Japanese translation. Isn’t there a way to search by excluding Japanese? The filter 'Show translations in:' allows to narrow down by language, but can’t I exclude Japanese? So I need to go through every sentence to see whether there is a Japanese translation or not."
* M\. clicks on the Add translation button of a Chinese sentence. "Let’s try to translate this one."
* M\. enters a translation.
* M\. selects "Japanese" as language.
* M\. clicks submit. The translation is added.
* M\. wants to add an audio recording and clicks the speaker button.
* "They kinldy explains how to add a recording. :-)"

#### Tatoeba wiki page "Contribute audio for Tatoeba"

* "What, it’s in English? Well, I don’t wanna read this. Let’s forget about it. Goodbye." M\. clicks the back browser button.

#### Sentences in Mandarin Chinese

* I ask M\. what she wanted to do. M\. decides to read the wiki page in Japanese using the Google translate button in her browser.

#### Tatoeba wiki page "Contribute audio for Tatoeba" (Google-translated version)

* M\. reads the page. "I understand what the page says, but eventually I don’t understand how to contribute audio. The only thing I understand is that audio is contributed by people."
* M\. goes back to the sentences list tab.

#### Sentences in Mandarin Chinese

* M\. clicks on the arrow icon of her translation

#### Sentence 8532850

* "So I can always edit sentences that have been added by me."
* M\. clicks the back browser button

#### Sentences in Mandarin Chinese

* M\. adds another translation. "Let’s add like five translations."
* M\. wonders if something will happen after she adds more translations, but she guesses probably not.

## Debrief

* I asked M\. about how she feels about adding translations.

> My impressions about adding these few translations is that I don’t feel anything. Using this website, it feels like nothing comes back even though I did something. There is no sense of accomplishment for contributors. After adding many sentences, I guess other people from the community will give me some feedback, like on the bulletin board. But it’s simply boring.

> The most important thing for language learners is motivation, but this website doesn’t maintain motivation. And how do get motivation? By getting a feedback of some sort. For example, let’s say I just registered. Then it says 'Thank you for registering! Alright, now let’s write a new sentence!'. It’s not really guidance, but if there were something like that, I’d easily understand what to do next, so I can directly move on to the next thing. However after I registered, nothing happened, so I didn’t know what to do.

> This website is really light (あっさりしてる), as both in pejorative and meliorative senses of the word.

* I asked M\. if she thinks Tatoeba could be of any use to her in as a teacher of Japanese.

> If audio recordings were to become the main part of the project, if the number of recordings were to grow, I’d introduce Tatoeba to my students as a tool they can use when they study alone since short sentences with audio are easy to study.

> Also, sentences are contributed by individuals, so quite a lot of sentences are not like what you can find on textbooks, and that’s interesting. For example, "さもなければ、死ぬよ。" I wonder when one would use that (laugh).

> People who study Japanese maybe just study for studying. A lot of them are in contact with the language only during the time they study it. It’s a pity because they are rarely in contact with anything else than textbook-like contents, as opposed to what is actually used for example in newspapers, magazines or novels. They don’t have much opportunities to see actual living language. While in places like Tatoeba, you have sentences like 彼女が飲むのが好きだ。(She likes to drink.). This is something you’d never see in a textbook. It means drinking alcohol. In a textbook, you’d only see instead 彼女がお酒を飲むのが好きだ。(She likes to drink alcohol.) And that’s because it was written by a native speaker. I like that.

> It depends on the way Tatoeba is used. There is an app I can introduce to students saying that they can learn new words by studying a little bit everyday. But in the case of Tatoeba, it would be difficult for me to introduce it to students. I wouldn’t know how to explain. There are other example sentence websites aimed at language learners, but I can’t really tell what’s the difference between them and Tatoeba. Well, if anything, I can say that Tatoeba includes native speaker sentences, but then again not all of them are. For example, that one, 彼は英語を大声で話す, I translated it into Chinese, but I doubt anyone would use a sentence like that in real life.

> I think Tatoeba would be easier for me to use if it made it clearer for what purpose students actually use Tatoeba. For example, let’s say I wanna study spoken/colloquial language, so it would be useful if I could search only into colloquial sentences. Or let’s say that I’m reading a novel, and there is a sentence I don’t understand, and I wanna see how to use a particular word. It would be useful if I could search only into sentences of written language. Right now I can’t do that. If the search results include a translation in my mother tongue, I can guess whether the original sentence is of written or spoken language, but honestly I don’t know if I can trust the translation on a website like this where many people contribute. Eventually it’s hard to draw a conclusion from the search results. I can’t really know whether a given word should be used in this or that way.

* I asked M\. to try to use the Translate sentences page from the top menu since she complained about not being able to exclude sentences already having a Japanese translation. She used the page but she completely ignored the criteria "Not directly translated into:" (直接訳があれば除外：). It turned out she thought it was about filtering literal translations, because the Japanese for "literal translation" (直訳) and "direct translation" (直接訳) are very similar. I explained her the concept of direct versus indirect translations and showed her the advanced search criterion "Link: direct/indirect/any". M\. said that she did understand the link criterion because the two words are displayed next to each other (直接訳・間接訳), but on the Translate sentences page, 直接訳 is used alone, so it’s confusing.

* About the Translate sentences page, M\. mentioned that it would be interesting to have a "popular sentences" sort, something like sentences with many translations.

## Identified problems

* Tatoeba doesn’t match the user needs:

* Usability problems:
  * Even though M\. used the top menu "Add translations" link as guest and saw a login page, once registered, she didn’t click that link to translate sentences. Instead, she used the "browse sentences" link to browse Chinese sentences and complained that you have to manually check if it’s already translated.
  * At some point during the exploration phase, using the new design, M\. clicks on a sentence text several times thinking it would do something but nothing happens.
  * M\. praised the presence of Pinyin on Chinese sentences, but she was surprised by the icon 漢 meaning traditional characters. "I see 漢, and I wonder what else there is. It would be easier for me to understand if the icon said 簡体字/繁体字."
* Feeling problems:

* Onboarding problems:

Note

The lines in green are the lines that have been added in the new version. The lines in red are those that have been removed.

Actions

• History
• Revert to version made the 09/02/2020, 11:16
• Go back to article's page