Difference between version 35 and 774

 Version at: 29/03/2013, 19:11 vs. version at: 09/02/2014, 18:01
 GSoC ideas for student projects
 ===============================
 This page lists example ideas for students who would like to take part in Google Summer of Code and be mentored by Tatoeba. To quote [GSoC FAQ](http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page#3._What_is_an_Ideas_list):
 &lt;blockquote&gt;
 &lt;p&gt;An Ideas list should be a list of suggested student projects. This list is meant to introduce contributors to your project's needs and to provide inspiration to would-be student applicants. It is useful to classify each idea as specifically as possible, e.g. "must know Python" or "easier project; good for a student with more limited experience with C++." If your organization plans to provide a proposal template for the students, it would be good to include it on your Ideas list.&lt;/p&gt;
 &lt;p&gt;Keep in mind that your ideas list should be a starting point for student proposals; we've heard from past mentoring organization participants that some of their best student projects are those that greatly expanded on a proposed idea or were blue-sky proposals not mentioned on the ideas list at all. A link to a bug tracker for your open source organization is NOT an ideas list.&lt;/p&gt;
 &lt;p&gt;You can check out the &lt;a href="http://community.kde.org/GSoC/2011/Ideas"&gt;Ideas list for KDE&lt;/a&gt; for Google Summer of Code in 2011 to get an idea of what we’re looking for in an ideas list. &lt;/p&gt;
 &lt;/blockquote&gt;
 If you're a student, you're invited to discuss any of these ideas, as well as propose your own. To contact us, use one of these:
 * [Tatoeba Wall page](http://tatoeba.org/wall/index)
 * Email/Google groups: [Tatoeba GSoC mailing list](https://groups.google.com/forum/?fromgroups=#!forum/tatoeba-gsoc)
 * IRC: [Tatoeba on #Freenode](irc://irc.freenode.net/tatoeba), [Webchat](http://webchat.freenode.net?channels=tatoeba)
 * XMPP: [Tatoeba conference room on chat.jabberfr.org](xmpp:tatoeba@chat.jabberfr.org?join)
 Current site
 ------------
 Extending current PHP site: programming in PHP, shell tools for maintenance (e.g.: better export scripts?), JavaScript?
 ### Better export scripts
 Currently CSV dumps are done weekly. They require the database to be switched into a read-only mode, take 5~10 minutes and do not contain some important information, like tag creator, comments etc. CSV dumps are important for people who cooperate with Tatoeba by creating additional tools, so their quality is vital for a healthy collaboration.
 **Deliverables**: An database export mechanism that:
   * Dumps all interesting information (everything that's currently in the data dumps plus modification history, sentence comments, the wall etc.)
   * Can create an incremental dump (faster dumps will allow making them more often)
   * Provides an interface for collaborators to get notifications about new dumps and allows automatic access.
   * (Advanced) Provides a stream of updates in form of web sockets or a similar mechanism.
 **Prerequisite knowledge**: a scripting language (Python preferred), PHP, MySQL.
+### Administrative Scripts
+The current site has experienced a couple of crashes and instabilities. We also would like to grow a number of users who are fully capable to administer it with ease and reliability. In order to ease administration and quickly recover from disaster, a number of scripts covering common administrative tasks are needed.
+**Deliverables**: Shell scripts that cover:
+ * backup
+ * adding new languages
+ * getting external services up and running
+ * updating the production site from the repository
+ * deployment on a real server from scratch
+ * monitoring the server and logging load and activity
+ * other necessary tasks
 ### API
 Tatoeba database is used either through the main website interface or through data dumps. Having a real API (like, something than can be called through AJAX calls and get machine-readable results) would allow providing real-time access for external applications.
 **Deliverables**: A web application that provides a set of API calls for data stored in current database. API should cover all data available through the current web interface, including sentence comments, wall comments, recently-added sentences and top recent contributors.
 **Prerequisite knowledge**: a web application language (Python or PHP preferred), MySQL, JSON or XML or any other data exchange format.
 ### Improvements in user interface for end users
 Tatoeba now handles several kinds of queries, but more is desired. Especially the translation interface needs improvements. Examples of desired types of queries:
 * Get all sentences in a given language by a given username not yet translated into a given language.
  * For example: Show me all English sentences by CK not yet translated into Japanese.
 * Same as above, but limited to sentences with audio.
  * For example: Show me all English sentences by CK with audio not yet translated into Japanese.
 * Get all sentences by native speakers of a given language not yet translated into my own native language.
  * For example: Show me all English sentences by native speakers not yet translated into Japanese.
 * Get all sentences in a given language with a certain tag not yet translated into a given language.
  * For example: Show me all English sentences with the tag “restaurant.”
 * Same as above, but limited to sentences by native speakers not yet translated into a given language.
  * For example: Show me all English sentences by native speakers with the tag "weather" not yet translated into Japanese.
 * Get all sentence in a given language under a certain length not yet translated into a given language.
  * For example: Show me all Japanese sentences less than 50 characters that aren't yet translated into English.
 * Same as above, but limited to native speaker sentences.
 * Same as above, but limited to sentences by a given username.
 * Get all sentences by native speakers of a given language that match a given search keyword that aren't yet translated into a given language.
  * For example: Show all English sentences with the word "mountain" that aren't yet translated into Japanese.
 * Same as above, but limited to native speaker sentences.
 * Same as above, but limited to sentences by a given username.
 **Deliverables:** Implementation of some (all?) of the above. Project might include additional queries. It would be highly desired to provide a generic way of adding new types of queries.
 **Prerequisite knowledge**: PHP, CakePHP.
 ### Show pronunciation in IPA for sentences
 IPA stands for “International Phonetic Alphabet” and is used to describe pronunciation of human languages in an unambiguous way. As such, it helps learning languages whose pronunciation rules are complex (e.g. English). Tatoeba could display IPA pronunciation for each sentence, basically the same way it currently displays pronunciation for Japanese using kana. One possible way of performing the task is to use an external library or application to prepare IPA annotations. For example, [eSpeak](http://espeak.sourceforge.net/) seems to be able to handle several popular languages and has an IPA converter.
 **Deliverables:** A mechanism that shows IPA pronunciation for some languages (chosen by student). This can be done server-side (as a standalone service or part of existing code) or client-side (using JavaScript). Mechanism should allow pre-generating pronunciation descriptions and should provide means to manually edit pronunciation later. Mechanism can rely on 3rd party tool to generate pronunciation descriptions.
 **Prerequisite knowledge**: web technology. some web application stack (PHP, Python or CppCMS preferred).
 New site &amp; CppCMS
 -----------------
 Helping Sysko with tatowiki, tatodb. Extending [CppCMS](http://cppcms.com/wikipp/en/page/main). As the new site is still mostly in plans, there are no specific project ideas for the moment. Please ask on the IRC channel for more information. Note that many projects in this category will have an experimental nature, and their scope highly depends on your skills.
 Standalone user tools
 ---------------------
 Work on [shtooka recorder](http://a4esl.com/temporary/tatoeba/shtooka/) (or swac-record), [tatoparser](https://github.com/qdii/tatoeba_parser), [katoeba](https://github.com/sadhen/katoeba) and similar tools. Create new tools for advanced contributors and common users, like apps for smartphones.
 ### Android/iPhone application
 iPhone users are about 12%, and Android users about 7% of the site visitors. It might immensely help them if they could use a dedicated application
 **Deliverables**: A smartphone application for easy access to Tatoeba. Example features:
 * Querying the online Tatoeba site
 * Adding sentences
 * Translating
 * Performing typical corpus maintenance tasks (linking/unlinking sentences, changing sentence language, tagging etc.)
 * Access to wall and sentence comments
 * Recording voice
 * Offline database access (more difficult!)
 Note: It is not expected to implement all of these features during a single GSoC event. Depending on your skills you might prepare a proposal for a basic set of features (if you don't have much experience in mobile development yet) or a more complex or targeted application (if you do have experience and want to prepare something more feature-complete).
 **Prerequisite knowledge**: Java and Android development or iPhone and iOS developement; using web services.
 ### Streamlined linking of multiple sentences
 Where multiple sentences in a source language have the same translation in the target language, make it easy to link those source sentences to the same target translation. Collecting the sentences that are likely to have the same translation could be as simple as presenting sentences in order of creation, since variants of a sentence that vary only in, e.g., the number of the pronoun (where the singular and plural forms of the second person map to the same word in English) are likely to be entered consecutively.
 **Prerequisite knowledge**: JavaScript, possibly Java, possibly SQL
 ### Help bots
 Just like [bots in wikipedia](https://en.wikipedia.org/wiki/Wikipedia:Bots), have bots to help doing maintenance and repetitive tasks such as fixing common mistakes, wrong flag etc. Like on wikipedia, users that are actually bots should be identified somehow on the website side. Ideally, create a library to interact with the Tatoeba website, that could be used as a base to create bots.
 Note: this idea would highly benefit from having a real API, which is another project listed here.
 **Prerequisite knowledge**: web
 Other ideas
 -----------
 External services, like [CK's Temporary Tatoeba site](http://a4esl.com/temporary/tatoeba/). Other ideas.
 ### SRS deck generator
 Spaced Repetition Systems such as Anki or Mnemosyne are a popular tool for learning languages. However, preparing a good SRS deck is a time consuming task. Therefore an automated way to generate a deck from a list of sentences (e.g. sentences on a Tatoeba list, sentences tagged by some specific tag etc.) would help language learners.
 **Deliverables**: an application (preferably a web-based one) that would use Tatoeba database (for example in form of a weekly CSV data dump) to create SRS decks for major flash card applications. Example features:
 * Generate a simple deck from a Tatoeba list, tag, search query.
 * Generate an N+1-style deck based on user's list of known words and Tatoeba database. (User gives a list of N words that he already knows. System chooses a new sentence where exactly one word is unknown, and the rest belong to the already known set. Therefore, N+1).
 * Generated decks have proper internal structure (e.g. for Anki decks: proper field scheme is used to store knowledge, so that editing is easy).
 **Prerequisite knowledge**: any web stack, however Django or CppCMS are prefered; Python or C++; knowledge about SRS.
 ### Browsable graph of sentences links
 Given a sentence, display as a [graph](https://en.wikipedia.org/wiki/Graph_%28data_structure%29) [like this](http://blog.tatoeba.org/2010/02/how-to-be-good-contributor-in-tatoeba.html#rule2) the linked sentences up to a given depth. The main purpose of such a graph is to show users how tatoeba is structured at a glance. Because the current interface doesn’t provide such a view whereas it’s important that the users understand the actual structure of Tatoeba. This idea could be freely extended to a complete interface allowing linking and unlinking with a click, filter by language, edit sentences, or whatever you can think of.
 **Deliverables**: a web application or a client-side JavaScript program that provides a graph view of a group of sentences, and allows manipulating them. Code can either use database directly or use existing or planned APIs.
 Note that this idea can be implemented as part of current code base in PHP or as an experimental service for new CppCMS site (preferred).
 **Prerequisite knowledge**: PHP, Python or CppCMS.

Version at: 29/03/2013, 19:11

GSoC ideas for student projects
===============================

This page lists example ideas for students who would like to take part in Google Summer of Code and be mentored by Tatoeba. To quote [GSoC FAQ](http://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2013/help_page#3._What_is_an_Ideas_list):

<blockquote>
An Ideas list should be a list of suggested student projects. This list is meant to introduce contributors to your project's needs and to provide inspiration to would-be student applicants. It is useful to classify each idea as specifically as possible, e.g. "must know Python" or "easier project; good for a student with more limited experience with C++." If your organization plans to provide a proposal template for the students, it would be good to include it on your Ideas list.

Keep in mind that your ideas list should be a starting point for student proposals; we've heard from past mentoring organization participants that some of their best student projects are those that greatly expanded on a proposed idea or were blue-sky proposals not mentioned on the ideas list at all. A link to a bug tracker for your open source organization is NOT an ideas list.

You can check out the <a href="http://community.kde.org/GSoC/2011/Ideas">Ideas list for KDE</a> for Google Summer of Code in 2011 to get an idea of what we’re looking for in an ideas list. 
</blockquote>

If you're a student, you're invited to discuss any of these ideas, as well as propose your own. To contact us, use one of these:

* [Tatoeba Wall page](http://tatoeba.org/wall/index)
* Email/Google groups: [Tatoeba GSoC mailing list](https://groups.google.com/forum/?fromgroups=#!forum/tatoeba-gsoc)
* IRC: [Tatoeba on #Freenode](irc://irc.freenode.net/tatoeba), [Webchat](http://webchat.freenode.net?channels=tatoeba)
* XMPP: [Tatoeba conference room on chat.jabberfr.org](xmpp:tatoeba@chat.jabberfr.org?join)

Current site
------------

Extending current PHP site: programming in PHP, shell tools for maintenance (e.g.: better export scripts?), JavaScript?

### Better export scripts

Currently CSV dumps are done weekly. They require the database to be switched into a read-only mode, take 5~10 minutes and do not contain some important information, like tag creator, comments etc. CSV dumps are important for people who cooperate with Tatoeba by creating additional tools, so their quality is vital for a healthy collaboration.

**Deliverables**: An database export mechanism that:

* Dumps all interesting information (everything that's currently in the data dumps plus modification history, sentence comments, the wall etc.)
* Can create an incremental dump (faster dumps will allow making them more often)
* Provides an interface for collaborators to get notifications about new dumps and allows automatic access.
* (Advanced) Provides a stream of updates in form of web sockets or a similar mechanism.

**Prerequisite knowledge**: a scripting language (Python preferred), PHP, MySQL.

### API

Tatoeba database is used either through the main website interface or through data dumps. Having a real API (like, something than can be called through AJAX calls and get machine-readable results) would allow providing real-time access for external applications.

**Deliverables**: A web application that provides a set of API calls for data stored in current database. API should cover all data available through the current web interface, including sentence comments, wall comments, recently-added sentences and top recent contributors.

**Prerequisite knowledge**: a web application language (Python or PHP preferred), MySQL, JSON or XML or any other data exchange format.

### Improvements in user interface for end users

Tatoeba now handles several kinds of queries, but more is desired. Especially the translation interface needs improvements. Examples of desired types of queries:

* Get all sentences in a given language by a given username not yet translated into a given language.

* For example: Show me all English sentences by CK not yet translated into Japanese.

* Same as above, but limited to sentences with audio.

* For example: Show me all English sentences by CK with audio not yet translated into Japanese.

* Get all sentences by native speakers of a given language not yet translated into my own native language.

* For example: Show me all English sentences by native speakers not yet translated into Japanese.

* Get all sentences in a given language with a certain tag not yet translated into a given language.

* For example: Show me all English sentences with the tag “restaurant.”

* Same as above, but limited to sentences by native speakers not yet translated into a given language.

* For example: Show me all English sentences by native speakers with the tag "weather" not yet translated into Japanese.

* Get all sentence in a given language under a certain length not yet translated into a given language.

* For example: Show me all Japanese sentences less than 50 characters that aren't yet translated into English.

* Same as above, but limited to native speaker sentences.

* Same as above, but limited to sentences by a given username.

* Get all sentences by native speakers of a given language that match a given search keyword that aren't yet translated into a given language.

* For example: Show all English sentences with the word "mountain" that aren't yet translated into Japanese.

* Same as above, but limited to native speaker sentences.

* Same as above, but limited to sentences by a given username.

**Deliverables:** Implementation of some (all?) of the above. Project might include additional queries. It would be highly desired to provide a generic way of adding new types of queries.

**Prerequisite knowledge**: PHP, CakePHP.

### Show pronunciation in IPA for sentences

IPA stands for “International Phonetic Alphabet” and is used to describe pronunciation of human languages in an unambiguous way. As such, it helps learning languages whose pronunciation rules are complex (e.g. English). Tatoeba could display IPA pronunciation for each sentence, basically the same way it currently displays pronunciation for Japanese using kana. One possible way of performing the task is to use an external library or application to prepare IPA annotations. For example, [eSpeak](http://espeak.sourceforge.net/) seems to be able to handle several popular languages and has an IPA converter.

**Deliverables:** A mechanism that shows IPA pronunciation for some languages (chosen by student). This can be done server-side (as a standalone service or part of existing code) or client-side (using JavaScript). Mechanism should allow pre-generating pronunciation descriptions and should provide means to manually edit pronunciation later. Mechanism can rely on 3rd party tool to generate pronunciation descriptions.

**Prerequisite knowledge**: web technology. some web application stack (PHP, Python or CppCMS preferred).

New site & CppCMS
-----------------

Helping Sysko with tatowiki, tatodb. Extending [CppCMS](http://cppcms.com/wikipp/en/page/main). As the new site is still mostly in plans, there are no specific project ideas for the moment. Please ask on the IRC channel for more information. Note that many projects in this category will have an experimental nature, and their scope highly depends on your skills.

Standalone user tools
---------------------

Work on [shtooka recorder](http://a4esl.com/temporary/tatoeba/shtooka/) (or swac-record), [tatoparser](https://github.com/qdii/tatoeba_parser), [katoeba](https://github.com/sadhen/katoeba) and similar tools. Create new tools for advanced contributors and common users, like apps for smartphones.

### Android/iPhone application

iPhone users are about 12%, and Android users about 7% of the site visitors. It might immensely help them if they could use a dedicated application

**Deliverables**: A smartphone application for easy access to Tatoeba. Example features:

* Querying the online Tatoeba site
* Adding sentences
* Translating
* Performing typical corpus maintenance tasks (linking/unlinking sentences, changing sentence language, tagging etc.)
* Access to wall and sentence comments
* Recording voice
* Offline database access (more difficult!)

Note: It is not expected to implement all of these features during a single GSoC event. Depending on your skills you might prepare a proposal for a basic set of features (if you don't have much experience in mobile development yet) or a more complex or targeted application (if you do have experience and want to prepare something more feature-complete).

**Prerequisite knowledge**: Java and Android development or iPhone and iOS developement; using web services.

### Streamlined linking of multiple sentences

Where multiple sentences in a source language have the same translation in the target language, make it easy to link those source sentences to the same target translation. Collecting the sentences that are likely to have the same translation could be as simple as presenting sentences in order of creation, since variants of a sentence that vary only in, e.g., the number of the pronoun (where the singular and plural forms of the second person map to the same word in English) are likely to be entered consecutively.

**Prerequisite knowledge**: JavaScript, possibly Java, possibly SQL

### Help bots

Just like [bots in wikipedia](https://en.wikipedia.org/wiki/Wikipedia:Bots), have bots to help doing maintenance and repetitive tasks such as fixing common mistakes, wrong flag etc. Like on wikipedia, users that are actually bots should be identified somehow on the website side. Ideally, create a library to interact with the Tatoeba website, that could be used as a base to create bots.

Note: this idea would highly benefit from having a real API, which is another project listed here.

**Prerequisite knowledge**: web

Other ideas
-----------

External services, like [CK's Temporary Tatoeba site](http://a4esl.com/temporary/tatoeba/). Other ideas.

### SRS deck generator

Spaced Repetition Systems such as Anki or Mnemosyne are a popular tool for learning languages. However, preparing a good SRS deck is a time consuming task. Therefore an automated way to generate a deck from a list of sentences (e.g. sentences on a Tatoeba list, sentences tagged by some specific tag etc.) would help language learners.

**Deliverables**: an application (preferably a web-based one) that would use Tatoeba database (for example in form of a weekly CSV data dump) to create SRS decks for major flash card applications. Example features:

* Generate a simple deck from a Tatoeba list, tag, search query.
* Generate an N+1-style deck based on user's list of known words and Tatoeba database. (User gives a list of N words that he already knows. System chooses a new sentence where exactly one word is unknown, and the rest belong to the already known set. Therefore, N+1).
* Generated decks have proper internal structure (e.g. for Anki decks: proper field scheme is used to store knowledge, so that editing is easy).

**Prerequisite knowledge**: any web stack, however Django or CppCMS are prefered; Python or C++; knowledge about SRS.

### Browsable graph of sentences links

Given a sentence, display as a [graph](https://en.wikipedia.org/wiki/Graph_%28data_structure%29) [like this](http://blog.tatoeba.org/2010/02/how-to-be-good-contributor-in-tatoeba.html#rule2) the linked sentences up to a given depth. The main purpose of such a graph is to show users how tatoeba is structured at a glance. Because the current interface doesn’t provide such a view whereas it’s important that the users understand the actual structure of Tatoeba. This idea could be freely extended to a complete interface allowing linking and unlinking with a click, filter by language, edit sentences, or whatever you can think of.

**Deliverables**: a web application or a client-side JavaScript program that provides a graph view of a group of sentences, and allows manipulating them. Code can either use database directly or use existing or planned APIs.

Note that this idea can be implemented as part of current code base in PHP or as an experimental service for new CppCMS site (preferred).

**Prerequisite knowledge**: PHP, Python or CppCMS.

version at: 09/02/2014, 18:01

GSoC ideas for student projects
===============================

If you're a student, you're invited to discuss any of these ideas, as well as propose your own. To contact us, use one of these:

Current site
------------

Extending current PHP site: programming in PHP, shell tools for maintenance (e.g.: better export scripts?), JavaScript?

### Better export scripts

**Deliverables**: An database export mechanism that:

**Prerequisite knowledge**: a scripting language (Python preferred), PHP, MySQL.

### Administrative Scripts

The current site has experienced a couple of crashes and instabilities. We also would like to grow a number of users who are fully capable to administer it with ease and reliability. In order to ease administration and quickly recover from disaster, a number of scripts covering common administrative tasks are needed.

**Deliverables**: Shell scripts that cover:

* backup
* adding new languages
* getting external services up and running
* updating the production site from the repository
* deployment on a real server from scratch
* monitoring the server and logging load and activity
* other necessary tasks