User experience test

Test performed by gillux on the 13/06/2019.

User background

  • Nationality: German
  • Job: teacher
  • Age: 34

M. is a teacher of German as a foreign language in Germany. Her students typically speak more than one language. She’s interested to use Tatoeba for her lessons. I quickly showed her the website already.

Test

  • Device: modern laptop with touchpad
  • OS: Windows
  • Browser: Chrome
  • Instructions: I did not give any precise instructions at first, to let M. go where she wants.

Exploration

Google

  • M. searches for Tatoeba and clicks the first result
  • M. is brought to the English version of the website. She will stick with the English version during the entire test. (Debrief revealed that using the German version would have helped understanding key words like logs.)

Main page

  • M. uses the from language dropdown to explore the languages by scrolling. "So many languages". She clicks on Corsican.
  • M. types in German in the To language dropdown.
  • M. would like to see sentences in one of the languages of the dropdown without specifying any keyword but she doesn’t know how to do that.
  • I tell her not to put any keyword and she submits the form.

Search results (keywords=, from=Corsican, to=German)

  • M. looks at the results and finds some of them funny, like "Wo ist das Brot?"
  • M. clicks on the (i) icon of sentence 4919833 to "maybe see the pronunciation?"

Sentence 4919833

  • "No, I see other languages."
  • M. is very much confused by the button named random. "What’s the meaning?" She clicks it.

Sentence 7565788

  • M. now understands she has been brought to a random sentence. She thinks it could be a good way to start a lesson, with something surprising/interesting/motivating.
  • M. is intrigued by the empty Tags block. She doesn’t understand the meaning. The block doesn’t contain any tags, just the link View all tags. She clicks it.

All tags

  • For some reason the page takes a long time to load and M. click the back button and then view all tags again. The page loads fast this time.
  • M. still doesn’t understand what are tags.
  • M. clicks on the 7 syllables tag.

7 syllables tag, first page

  • M. still tries to understand the meaning of "tags" by comparing the sentences. She scrolls the page up and down multiple times. She doesn’t understand why the sentences are in English only.
  • M. goes back to the All tags page.

All tags

  • M. feels a bit "overwhelmed" by the list of tags.
  • M. clicks on the sports tag because it looks interesting.

Tag sports

  • M. thinks it could be interesting to have sentences grouped in topics like sports, so that her students could start with a topic. However she still wonders "how does it works?"
  • M. sees the filter by lang block and thinks it’s interesting.
  • M. tries to filter by German. German sentences appear. She tries to filter by Tatalog or Swiss German, but it gives no results. She is happy about seeing dialects like Swiss German.
  • M. is still confused about tags but thinks it may be some sentences that are collected.
  • M. wants to see sentences in Swiss German and she types in swiss german in the search bar and press enter.

Search results (keyword=swiss german, from=Corsican, to=German)

  • No results.
  • In the search bar, M. removes swiss german as keywords, selects Swiss German in the From dropdown and asks me how not to set any target language. I tell her to select Any language.

Search results (keyword=, from=Swiss German)

  • M. is amused by the result "Broscht!", but mentions there is no German translation for it (actually there is, but it is hidden under the Show 81 more translations button).
  • We decide to register to add translations. She clicks the Register top link.

Registration

Register

  • M. fills the form. Her choosen username is already taken so she appends a number to it.
  • M. is intrigued by the We need to make sure you are human question. She says the example is confusing because the answer a.b.c only includes three characters, not five. Eventually she understands the dots are part of the answer.
  • M. submit the form successfully and is brought to the home page.

Home

  • The greeting message advices M. to fill out her profile. She decides to "follow the good advice".

Profile page

  • M. clicks the button Edit profile.

Edit profile

  • M. fills her name and country. She likes that she can type G to select inside the country drop down, because there are some websites where you need to scroll the list forever.
  • M. submits the form and is brought to the profile page.

Profile page

  • At first M. doesn’t understand where to specify her language, but she eventually finds the Add a language button after scrolling down. She clicks it.

Add a language

  • M. adds English. She doesn’t think it’s necessary to add details because she has no dialect, while that’s the interesting information for other users.
  • M. adds French and takes some time to fill the details field to specify where she learnt it from, because that’s useful for other users.
  • Same for Spanish.

Profile page

  • M. doesn’t understand the Stats block because she’s not familiar with this word. "Is is status?" After looking it up in an online dictionary, she understands it is statistics.
  • M. sees the link Translate X's sentences and thinks it’s interesting because it means other users can click it to translate her sentences.
  • M. now wants to find other German members.

Finding German members

M. hover the Community top menu and is intrigued by the Wall entry. She has no idea what it is. She clicks.

Wall (first page)

  • M. thinks it’s a kind of chat. She looks at the messages.
  • M. is amused by a user’s profile picture that shows an open vagina. As the message contents is hidden, she understands it is supposed to be censored.
  • M. says the Latest messages block is not really important for her, she’d rather not see it.
  • M. likes the block "all languages are equal" but she wonders if it’s practical for communication.
  • M. continues to look at the messages in order to find a German one. She clicks on the last page pagination link.

Wall (last page)

  • M. continues to look at the messages in order to find a German one. She clicks on another page number.

Wall (some other page)

  • M. figure out all the messages are in English and gives up using the Wall to find German contributors.
  • M. clicks the list of all members menu entry

List of all members

  • M. cannot find where the users are from and decides this page is not useful for her.
  • M. clicks on the languages of members menu entry

Languages of members

  • M. doesn’t quite understand the table. She is confused by the column level=0 and clueless for a while until she figures out there is a legend on the right.
  • M. still doesn’t understand the column level=0 and level=unknown. "Why people would add a language they have almost no knowledge in?" (Debrief revealed that she assumed the only reason one would add a language is to translate from it, so it doesn’t make sense to add a language in which you know almost nothing.)
  • M. clicks the link German

Members with language German in their profile

  • M. understands the users listed here all speak German as native.
  • M. is amused by the username Trinkschokolade and clicks it.

Trinkschokolade’s profile

  • M. would like to click on "German" in the language list to see Trinkschokolade’s German contributions, but there is no link.
  • In the Stats block, M. is confused by the number of Contributions because it doesn’t add up with the other numbers.
  • M. clicks on the link Sentences

Trinkschokolade’s sentences

  • M. is a bit confused because the sentences are pretty unrelated/random. She wonders what is Trinkschokolade’s approach when contributing sentences. She goes to page 6 and keeps trying to find some consistency in that list of sentences.
  • M. mentions 3 sentences that are almost synonyms but she’s not quite sure why.
  • M. mentions that the sentences are not consistent because some include quotes while others do not. (She’s actually referring to dialogs vs. normal sentences.)
  • M. clicks the link Collections

Trinkschokolade’s collection

  • The page is empty but the block "This feature is currently deactivated." shows on the right. M. decides to activate the option in her settings.
  • M. clicks the Settings menu entry

Collection feature

Settings

  • M. sets her profile as public. She doesn’t quite understand the meaning of the Remember the last list... option.
  • M. activates the collections feature. "I can have a collection of useful sentences for my lesson."
  • M. likes the option Display "(native)" next to username and activates it.
  • M. activates the clipboard button.
  • M. doesn’t understand what is the old/new design option about.
  • M. clicks Save.
  • M. browses the My menu and wonders what is the difference/relation between My collections and My favorites. She clicks My favorites.

Own Favorites

  • The page is empty.
  • M. is intrigued by the Logs link. She has absolutely no clue about its meaning. She look up the word logs in an online dictionary, but it doesn’t help. She clicks the Logs link.

Logs

Own Logs

  • The page is empty.
  • M. is very much confused about the logs.
  • She decides to check another user’s logs to understand better.
  • She goes to the Languages of members page, then Native speakers, then Languages of members again, then clicks German, then clicks on the user raggione, then clicks on Logs.

Logs of raggione

  • M. is still confused. She clicks the (i) of the first log record and is brought to sentence 7976858.

Sentence 7976858

  • M. is more and more confused. "It’s a sentence, not a log. So the sentence is a log?"
  • M. clicks on the button « previous and she is brought to sentence 7976857, a Chinese sentence.
  • M. is very confused. "I thought I’d go to the previous page…" She click « previous again. She is brought to sentence 7976856, another Chinese sentence. Still very confused.
  • I explain her she has to use the back button of her browser to get out of where she is. She goes back to the German sentence 7976858.
  • M. notices the clipboard button and clicks it. She opens a Google document and paste the sentence. "OK, that’s very useful. I can put the sentences on my worksheet."
  • M. decides to translate sentences.

Translation

  • M. clicks the Translate sentences menu entry.

Translate sentences

  • M. fills the form: sentences in English not yet not directly translated into German. Then, she hesitates about the sort order. "Short sentences are maybe easier to translate? Last created makes sense too."
  • Eventually M. decides that it doesn’t really matter and leaves it to the default: random.

Search result

  • M. wonders how to translate the first sentence. "It’s difficult to translate."
  • M. clicks the Show sentence details link.

Sentence 68680

  • "I could post a comment to ask for some context."
  • M. hesitates about posting a comment because she believes comments are public, but eventually decides to do so.
  • M. writes a comment and then reads the Good practices list. About Say "welcome" to new users, she mentions that she cannot know if a user is new or not. About Use private messages to discuss things unrelated to the sentence., she says her comment is related to the sentence so it’s okay. She submits the comment asking for more context.
  • M. clicks the back button of the browser.

Search result

  • M. is surprised to see a different list of sentences. "Where is my sentence?"
  • M. clicks on another sentence’s page link.

Sentence 2492237

  • M. hover the sentence menu buttons and click on the Translate button
  • M. writes a German translation.
  • M. opens the language dropdown and wonder why there is an autodetect entry. She selects German. (Debrief revealed that’s because she assumed people translated only into one language.)
  • M. is happy about having added her first translation. However, she is disappointed that translation is not performed in a structured way. She’d like to see a beginning and an end. "This is too random." It’s not satisfying for her to translate random sentences, she’d be more motivated to translate if, for example, other users would be in need for specific translations of certain sentences in certain languages.
  • M. discovers the top-left logo is a link and she clicks it.

Homepage as logged-in user

Homepage

  • M. is disappointed about the homepage. She finds it confusing. She wonder why it only shows features and why there is this random sentence.
  • M. expected the homepage to tell her more about the project: its philosophy, its history, its motivations, the preservation of endangered languages (which I all mentioned to her before the test), some guidelines, the fact that it is an NPO (which is really cool)…

Debrief

  • Even thought M. took some time to hover the sentence menu buttons, she didn’t realize the rating buttons (OK/NOK/unsure) were clickable. Reasons include: they use a white-on-black design while other buttons are black-on-white; they stand apart from other buttons.

Identified problems

  • Usability problems:

    • The human verification question of the registration page is hard to understand.
    • When posting a comment, we ask to say "welcome" to new users, but it’s not obvious if a user is new or not
    • [Solved] The "translate sentence order=random" page brings a different order after going to a sentence details and clicking on the back browser button.
    • [Solved] The Collection feature has a misleading name ("I can make a collection of useful sentences for my lesson")

  • Home page:

    • The homepage, especially while logged in, doesn’t give enough information about Tatoeba.
    • The home page and the search bar force M. to discover Tatoeba through a search form while M. is not interested in searching yet. Instead, M. would like to:
      • explore the list of languages (she does that by scrolling the language dropdown)
      • explore language pairs (I need to explain her to leave the keywords field blank)
      • explore sentences of a given language (I need to explain her to set the language dropdown to "Any language")

  • Onboarding problems:

    • M. has a hard time understanding what tags are and how they work.
    • going from the welcoming banner to the "add a language" button require unnecessary research
    • the meaning of the Logs page is difficult to understand
    • M. got a wrong idea about the purpose of having a language listed on the profile page.

  • Information is difficult to find:

    • M. has a hard time finding speakers of her mother tongue, because everyone speaks English on the Wall, and the "List of all members" doesn’t help. The "Languages of members" page eventually helps, but that is not obvious.
    • From a member’s profile page, M. cannot see sentences contributed in one of the languages the member listed.

  • Makes the user feel bad:

    • On a sentence page, the "previous / random / next" block is very confusing.
    • The "view all tags" page feels overwhelming.

  • Tatoeba doesn’t match user needs:

    • Translating sentences feels too random, M. wants to translate for users in need for specific translations of certain sentences in certain languages.

Contents

Actions

Article available in: