Design for Colloborative Translation Tool

Hi all,


This thread is the continuation of http://community.elgg.org/pg/forum/topic/808710/collaborative-internalization-for-elgg/.

Here is my initial design for the colloborative web based Translation tool.

Approach:

  1. The user will upload the plugin in .ZIP format to this webbased tool. The same will be done for CORE too.
  2. The tool will automatically unzip the file and reads the manifest.xml file to get the Plugin details. The details will be added to the PLUGIN_MASTER table.
  3. Then the php files inside the languages folder will be scanned and will be read individually.
  4. Each and every language key and value of each language file will be parsed and loaded in to the PLUGIN_STRINGS table.
  5. Then all the loaded strings will be shown in a new page for the users to translate each string in the language that they need.
  6. English will be considered as the standard language and will show up the string for other languages with the % of translation completed.
  7. User can add a new language for the plugin, if it is not already translated in that language.
  8. For each custom language selected, all the string will be displayed. User can report a particular string if the translation is not correct. For this string, the column str_marked will be TRUE.
  9. Other users can validate the strings for which the str_marked column is TRUE to come up with the better translation.



PLUGIN_MASTER Table:

plugin_id      : Integer,Autogenerated, Primary Key
plugin_name    : Text(30)
plugin_version : Text(10)
plugin_author  : Text(30)
plugin_tms     : CURRENT TIMESTAMP


PLUGIN_STRINGS Table:
plugin_id      : Integer,Autogenerated, Primary Key
str_lang       : Text(5)
str_key        : Text(50)
str_value      : Text(200)
str_marked     : Boolean - Default value of FALSE
plugin_tms     : Insert TIMESTAMP
plugin_updt_tms: Last Update TIMESTAMP


PLUGIN_LANGUAGES Table:
iso_lang       : Text - Holds the valid ISO language names

  • Yeah. I had a look on Launchpad.

    For the idea proposed by me, there was not much positive feedback. So I have to stop my work n that.

    It will be better if ELGG community has the translation framework on its own. Finally its the Core developers who need to decide on this.

  • Here is the ticket relevant to this: http://trac.elgg.org/ticket/3342

    The full story is this:

    • Up through Elgg 0.9, Elgg used gettext for translation. This is a standard tool, used by a lot of open source packages. It also has a lot of support from online, collaboration tools for translation.
    • The core developers switched to a custom approach. The claim was that it would be faster. I was not involved with Elgg. I don't know whether it is faster. I also don't know what other issues led to the change.

    Launchpad was used for these early versions. The core developers never shut down that site. I just requested that the Elgg project there be removed. (Apologies to Daniel on this. I had no idea that site was still up.)

  • Thanks for the updates Cash. You were clear in explaining things as usual. :)

    What I feel personally is that, even after ELGG implements gettext function there should be a platform like Launchpad for all to contribute towards translation.

    I am ready to be part of this activity to make ELGG reach many people in multiple languages.

  • @purus  I´m with you in this work, ok?

  • Thanks for your words Cerceau. Lets get the suggestion from Cash how we could contribute.

  • Some sort of framework to make this possible would be ideal. I'm equally ready to contribute language files.

    A custom plugin for community site would be a great idea. Or an expansion of the current plugin repo. I'm still working on another project at the moment, but would be happy to look into this at a later date.

    Found this interesting thing about editing subfiles in zipfiles. Seems to me that this kind of approach would be pretty quick and painless. Expand the plugin repo (for each release), to allow users to add a language file to it. Then plugin owner can update their plugin onsite by adding the new language file to it or replacing a language file already in it.

    Could even have a "form" view that outputs the English language file and allows editing & saving, so don't even need to download to computer and then reupload later as new language file.

  •  

    Contributing with some point to discuss.

     

    I´ve seen two ways to deal with structure of  translations archive.

     

    • The original one providy by the core team. 

    $english = array(
                'translationbrowser' => 'Translation Browser',          add_translation("en",$english);

     

    • And the new way developed by Evans (inside 'theme facebook').

    <?php
    add_translation('en', array(
        'annotation:group_topic_post:value:placeholder' => 'Leave a reply...',
    ));

     

    The translation software use to put the sentences in XML code to help the reuse of words/sentences (prevent the mistakes in translation with the same sequences).

     

    Some times, we found an old translation.  To use it, we need to take the archive and compare both similar field by field. Adding, deleting or changing elements.

    • The input process of old files could be automatic;
    • It´s helpfull to put a automatic search to find when the fields matches and when the fields belong to old version could be use to actual translation

     

    Collaborative system could let the people

    • contribute at any time
    • vote in the 'best choice' (or better translation)
    • automatic provide a zip/rar/tar file anytime based on the 'better translation'.
    • downloada file before selection by the user to only the core system translation or to a specific(s) plugin(s) file(s) with/without the core translation.
    • upload a file to contribute with a specific plugin or core translation.

     

    Automatic system could:

    • use the translation tools in the web (like translate.google) to help/assist the translation process. (please, I ´m remember an old suggestion using this direction inside the foruns of Elgg community; we need to looking for it)
    • suggest some translation to 'start the translation process'
    • find the different terms used to express the same content

     

    Elgg definitions use Languages according to ISO 639-1. This is not convenient to be use to regionaly languages, like: 'portugal portuguese' and 'brazilian portuguese'. We need to improve this tables.

     

     

  • //````````````````````````````````````````````````````````````````````````````````
    I've coded up some utility stuff to analyze the language files
    in an install and extract the most primary language attributes.

    //```````````````````````````````
    LANGUAGE-DATA
        BLOCK[]                {Elgg|PlugIn}
            LOCATION[]        {chr 64}
            COUNTRY[]        {chr 4}
                KEY[]        { chr X}
                TEXT[]        {chr X}

    The different coding methods :-
    1    add_translation("en",$english);
    and
    2    add_translation('en', array(
            'annotation:group_topic_post:value:placeholder' => 'Leave a reply...',
          ));
    3    The language file naming inconsistencies.

    -- make it more difficult to design code that will parse the 
    content easility - without having to write more involved code
    to handle all the coding stye variants. 

    The style #1 was the easiest to cater for in the code logic.


    //````````````````````````````````````````````````````````````````````````````````
    The end-result was something like :-

    custom_index           en english  C:/xampp/htdocs/Elgg1810/mod/custom_index/languages/en.php
    dashboard              en english  C:/xampp/htdocs/Elgg1810/mod/dashboard/languages/en.php
    dds_loginhook_         en english  C:/xampp/htdocs/Elgg1810/mod/dds_loginhook_/languages/en.php
    developers             en english  C:/xampp/htdocs/Elgg1810/mod/developers/languages/en.php
    developer_debug_views  en english  C:/xampp/htdocs/Elgg1810/mod/developer_debug_views/languages/en.php

    -- reporting the PlugIn Name, language code(s), file locations (++ language strings) of
    the whole Elgg install. That extracted data could then be used to create a database of
    language translations. With the proper logic wrapped around that database - a common multi
    user update-able and versioned language store can then be built - to satisfy the various
    features tht have been mentioned so far for such a language 'Collaborative Translation Tool'.


    //````````````````````````````````````````````````````````````````````````````````

  • @DDS: I have done that too and it works perfectly as of now.

    1) Upload a plugin zip file.

    2) Extract the zip file, read the manifest files for getting plugin details.

    3) Read the files inside the languages folder.

    4) Read the individual files and parse the language strings. Store them in the database.

    www.iyaffle.com/elgg-i18n

    (The link is down currently due to server issue).

  • oh well - if it works perfectly, no need for further improvements except to get the server issue fixed ;-) ?

Feedback and Planning

Feedback and Planning

Discussions about the past, present, and future of Elgg and this community site.