Welcome to palaso.org

Testing the New WeSay Collaboration Features

Author John Hatton

Since we first introduced WeSay a couple years ago, we’ve heard one request over and over: “Make it so that multiple people can collaborate on the dictionary”.  The need for this is actually greater than it was for ShoeBox, ToolBox, Lingualinks, or FieldWorks.  These other programs were primarily for linguists, and linguists working on smaller languages rarely have the luxury of a several partner linguists.  In contrast, it’s very likely that multiple members of the language community will want to contribute to a WeSay project.  In addition, WeSay is designed to be collaboration between and advisor and a native speaker.  They will most always have different computers and live in different places. All that to say we’ve known this was important for years, but it’s quite a challenge to get the collaboration powerful enough and yet simple enough for WeSay users.  Building this system, known as Chorus, has consumed the lion’s share of our new feature time over the last year.

There’s another reason why we’ve been slow to put this feature out, and why we still (as of December 2009) are only slowly, cautiously introducing it.  It turns out that when we add a system which hooks all the team members together, we’ve just dramatically increased the potential for the entire team to suffer from one person’s mishap.  For example, imagine that one user deletes or renames a critically important file.  If that change is transmitted to the rest of the team, none of them will be able to keep working.  Some of these problems have proved hard to predict ahead of time.  As we identify such scenarios, we attempt to add protection against them.

We understand that people are tired of waiting.  And we could use more feedback to get this ready for Prime Time.  So I’m writing this blog for those of you who are willing to spend some time finding out if WeSay’s collaboration features are solid enough for your team.  IF you find that they are, then great, you can start benefiting from them.  If you find that more work is needed, and you tell us about it, then we’ll be able to better prioritize our efforts to get you what you need.

Note: this blog entry is a work in progress… I expect to improve it over the coming weeks, as people try it out and give me feedback.

Eventually, getting started with WeSay and Language Depot needs to be a lot more simple than it is today. Please don’t hesitate to ask for further help.

Get a LanguageDepot.org Account for the Manager

Go to Language Depot and create yourself an account.

Please don’t use the same password you use for anything important… WeSay is NOT going to be careful about keeping your password well-hidden.

 

Get a LanguageDepot.org Project for the Language

Write to admin@languagedepot.org.  Please provide the following information:

• The name of the account you created in the previous step
• The name of the project.  Normally, the name of the language works well for this.
• The ISO 639-3 code for the language.  Easiest way to find that is to Google for “ethnologue thelanguagename”.

You will be the initial “manager” of the repository.  You will be able to assign contributors to the project, and turn features of the web site on and off.

People you have not added to the project will not be able to access your data.  However, we wouldn’t pretend to promise any real “security”.  If you need that, it’s perfectly OK to use a Mercurial server somewhere else… you aren’t tied to LanguageDepot.org.

Unless you tell us otherwise, we’ll assume it is ok for us to occasionally look at the files in your repository project for the purpose of fixing a problem for you or seeing how the collaboration features are being used in real projects.

 

Get LanguageDepot.org Accounts for the contributors

Create accounts for the contributors just like you did for yourself.  Then go to the settings for the project and assign each of those people as contributors to the project.

Get the Data Together

It’s important that there is a single, up-to-date copy of the dictionary when you first put it up on Language Depot.  If there is currently only a  single person working on the dictionary, you need to get their project, and delete the project from their computer.  That does two good things: ensures they don’t keep working on it, and ensures that they will be using the proper version of the project later.

Make sure you backup the WeSay project from the stable version of WeSay you’ve been using (e.g. version 0.6).  If you decide that the development version (0.7) isn’t ready for you yet, you’ll want to go back to 0.6, and the files are not backwards-compatible.

If there are multiple copies of the dictionary out there, you need to do that for each one of them.  That is, get the project, remove it from their computer.  You have an extra step in this case, which is to merge the entries together.  The simplest case is where the users have been working on completely different sets of words, with no overlap. That is, they each started with completely empty dictionaries, which have never once been merged together.  In this specific case, you can merge them by hand.  Do that by opening each .lift file and copying all the <entry>…</entry> chunks of one file in next to the <entry>…</entry> chunks of the other file.    Open in WeSay to make sure you didn’t mess the lift file up.

In the more general case, you will want to merge them together using FieldWorks Language Explorer (FLEx).  To do that, follow these steps:

1) Create a new project using FLEx.

2) Import each .lift file into the project, one at a time, until you have a nice combined dictionary.

3) Export the dictionary as a .lift file.  That file needs to be placed in the WeSay project you will be pushing up to Language Depot.

Get the Most Recent Version of WeSay

As of December 2009, these features are too new for us to support a large number of people using them. Therefore, you have to go “to the factory” to get it.  To get instructions on where to get it, simply email me (hattonjohn on gmail).

Install TortoiseHg

WeSay uses the Mercurial version control system to support sending data around and keeping a history of it.  For Windows, the easiest way to get Mercurial is from a free system named “TortoiseHG”, which you can download from here.

Push the project up to LanguageDepot

Ok, once you have a single dictionary folder with the whole team’s data, it’s time to do the initial push up to LanguageDepot.  At this point, WeSay can’t do this initial push for you.  The easiest way is to open a good ‘ol “command prompt” window.  If you don’t know how to do that, probably you’ve already enlisted a more technical person in previous steps. Get them to do this step too.

First, make sure Mercurial (hg) is properly installed. Type “hg version <return>”. You should see a little version message:

Now, change the directory to be that containing the WeSay dictionary.

Next, push the project up to Language Depot.  Here, instead of “wagi”, you’d type the ISO 639-3 language code you used when setting up the project.

You will be asked to enter the user name and password you used in setting up your Language Depot account.

Pull the project down to your computer

I recommend that you now pull the project right back to your computer as if you had never had it.  So if the project is already right where you want it, move it, rename it, or back it up and delete it.  Now, run the WeSay Configuration Tool, and click “Get From Internet”:

Here comes the most un-cooked part, where we make you type a whole bunch of stuff into one big scary URL:

So, if your account on Language Depot is “lucy_loo” with password “v9isual9”, and the ISO code for the language is “foo”, then you’d enter:

http://lucy_loo:v9isual9@hg-public.languagedepot.org/foo

Give the project a name, and then click “Download”.

Turn on Collaboration

Ok, it’s dumb that this isn’t automatic yet, but you need to enable Send/Receive:

This will cause a button to show up on the dashboard:

Clicking it there (or even while you’re still in the WeSay Configuration tool) will show a dialog like this (sorry about that obvious bug in the text at the bottom):

Clicking “Internet” starts the synchronization:

Note: when WeSay detects that some changes were pulled down from the internet, it closes down and restarts itself so that it has a nice clean start with the new data.  We plan to make this unnecessary in some future version.

Pull the project down to the computers of the rest of the team

Now do the same for each member of the team, as you get Language Depot accounts for them.  If you have limited internet connectivity, you can speed things up by instead using a USB flash drive.

Like most things in WeSay, you, the Advisor, need to turn on the collaboration features which are appropriate for your project.  Some of this could eventually be done automatically, but for now, you need to do this for each user (or do it for one and then copy their “.userConfig” file for everyone else, renaming each copy to match each person’s user name).

“Advanced History”

There are two optional tasks you can enable if you want:

These show up on the dashboard under the “review” section:

The history show you all the changes the team has made:

It’s labeled “advanced history” because, frankly, this is more complex than what we expect many WeSay users to handle.  Use your own discretion. It may be that you, the advisor, will want this enabled in your configuration, but not in that of the rest of the team.

The Notes Browser

Collaboration is more than just sharing changes.  It turns out that as soon as you start working with others using Send/Receive, you find you want to write notes to them.  So we added the ability to attach questions to lexical entries, and to carry on conversations about the question.  Future versions will add other kinds of notes, and allow them to be attached to particular fields, not just whole entries.

In the following screenshot, notice the circled buttons.  We’d click the left-most to add a new question to this entry.  The other button represents a previous, unresolved question. A click on it brings up a dialog box in which we can read and answer the question.

Ok, but how do you find which entries have unresolved notes?  WeSay 0.7 also introduces the Notes Browser, which lets you find and interact with notes from all over the dictionary:

In addition to Questions, WeSay currently supports just one other kind of note, the Merge Conflict.  These are created by the automatic merger when two team members edit the same field at the same time. Unlike traditional version control systems, Chorus (the engine we’ve written to do all this) doesn’t stop the merge and force you to deal with the problem immediately.  Instead, it makes its best guess as to what to do, then creates a Merge Conflict Note which the team can read and deal with when it is ready.

If you don’t like what the merger did, go the entry and make whatever changes are necessary. Then click the “resolved” box to show that this has been dealt with. Or if you need to discuss what to do with a teammate, add a new message to the note. In the following screenshot, I’ve highlighted the hyperlink at the top, and the Resolved box at the bottom.

Ok, sorry, I know that’s a lot to absorb in on sitting. When we’re beyond this alpha testing period, I imagine I’ll reintroduce all of this pieces, with greater detail (though still lacking pedagogical skill :-)   ).