palaso.org

XML indentation in .Net

Tim — Mon, 22 Feb 2010 06:25:34 +0000

Upon integration of ) to manage WeSay’s XML encoded .lift file, the exact format of that file has taken on a new importance. Because Mercurial (and Chorus at this point) uses a standard line diffing tool to express the difference between two revisions line breaks, indentations and other white space have suddenly become an issue where they normally are not in XML documents. As it turns out formatting XML in .net is not entirely trivial. Though the XMLWriter and XMLReader as well as their respective XmlWriterSettings and XmlReaderSettings have various switches for enabling and disabling indentation and linebreaking on attributes, these are bound together by subtle interactions which I hope to shed some light on in this post.

First some background:
The indentation and whitespace in a given XML file can be of interest for at least two reasons:
- line differs typically care about whitespace and the name itself bears witness to the importance of newlines.
- readability. It’s much easier for a humans to read a nicely formatted XML file.

In WeSay the .lift file is frequently created from two seperate files. First, a valid .lift file and secondly a .lift fragment file. Each time an entry is added or modified these two files are merged to form the new .lift file. For this reason we are interested in the interaction between an XmlReader and the XmlWriter that outputs said readers data.

As an example we will use some very simple XML rather than an actual lift file as that construct is unnecassarily complex for this discussion.
Here are the two source files we will be working with:

File 1:

    at1=”at1″
    at2=”at2″>


        at1=”3at1″
        at2=”3at2″ />


File 2:

    at1=”at1″
    at2=”at2″>

              at1=”3at1″
        at2=”3at2″ />


Here is our envisioned result:

Resulting File:

    at1=”at1″
    at2=”at2″>

              at1=”3at1″
        at2=”3at2″ />


    at1=”at1″
    at2=”at2″>

              at1=”3at1″
        at2=”3at2″ />

All of these files have been written with indentation and new lines for
each element and attribute. This makes for ok readability and should
keep our diff files nice and small.

The first thing we are going to do is to see what happens when we start with completely unformatted input files, default readers and a default writer:

File 1:

File 2:

And the code goes something like this:
XmlReaderSettings readerSettings = new XmlReaderSettings
                                                   {
                                                       ConformanceLevel = ConformanceLevel.Fragment
                                                   };

XmlWriterSettings writerSettings = new XmlWriterSettings
                                                   {
                                                       ConformanceLevel = ConformanceLevel.Document
                                                   };

XmlReader reader0 = XmlReader.Create(stream0, readerSettings);
XmlReader reader1 = XmlReader.Create(stream1, readerSettings);
XmlWriter writer = XmlWriter.Create(stream2, writerSettings);

while (!reader0.EOF)
{
writer.WriteNode(reader, true);
}
while (!reader1.EOF)
{
writer.WriteNode(reader, true);
}

With these settings the resulting file looks like this:

Just one long line… pretty much the worst case possible for a line diffing tool and for reading. So let’s spruce it up a bit and add some formatting to the
result file by changing the WriterSettings a bit:

XmlWriterSettings writerSettings = new XmlWriterSettings
                                                   {
                                                        Indent = Indent,
                                                        NewLineOnAttributes = true,
                                                        ConformanceLevel = ConformanceLevel.Document
                                                   };

Resulting file:

    at1=”at1″
    at2=”at2″>

              at1=”3at1″
        at2=”3at2″ />


    at1=”at1″
    at2=”at2″>

              at1=”3at1″
        at2=”3at2″ />

*sigh*… beautiful.
But being geeks we just can’t halp but fix something that ain’t broke. So
inspite of this beautiful result we now want to try and fix up the two
source files. This isn’t entirely unreasonable considering you may want
to look at the source files while debugging and it would be nice if they
were a bit more legible. So just for kicks, let’s see what happens when
we put a single line break in a source file.. say after the
element.

File 1:

at1=”"3at1″” at2=”"3at2″”>

Resulting File:

    at1=”at1″
    at2=”at2″>

        at1=”3at1″
        at2=”3at2″ />

    at1=”at1″
    at2=”at2″>

              at1=”3at1″
        at2=”3at2″ />

?!!?
what happened?! Not only do we have a line break after the
element, but also the element and the closing
element are not indented!!
This brings us to our first
interesting observation: Whitespace in a source document
causes the writer to ignore it’s Indent Attribute until the containing
element of the whitespace (in our case ) is
closed. this is true of whitespace such as “spaces” as
well. Here is the resulting file if I substitute the newline of our
last example with a simple space:

    at1=”at1″
    at2=”at2″>
            at1=”3at1″
        at2=”3at2″>

    at1=”at1″
    at2=”at2″>

              at1=”3at1″
        at2=”3at2″ />

Interestingly, you’ll notice that the NewLineOnAttribute Property of the XmlWriterSettings is NOT ignored. This is even more interesting when you consider that this property is ignored UNLESS the Indent Property is TRUE. here it is straight from the horses mouth (i.e. ): This setting has no effect when the Indent property value is false.

Ok..
so we’ve established that whitespace is an issue. The easiest way to
get around this is to instruct the reader to ignore whitespace so that
the writer doesn’t get to clever on us:

XmlReaderSettings readerSettings = new XmlReaderSettings
                                                   {
                                                        IgnoreWhitespace = true,
                                                       ConformanceLevel = ConformanceLevel.Fragment
                                                   };

So now we are back on track and looking good! To celebrate, let’s tell the
world how happy we are! Let’s write a string into our first file that
will proclaim our joy! Of course we will do this without spaces.. just
in case.

File 1:
I’mSoHappy at1=”"3at1″” at2=”"3at2″”/>

Resulting file:

    at1=”at1″
    at2=”at2″>
    I’mSoHappy        at1=”3at1″
        at2=”3at2″ />

    at1=”at1″
    at2=”at2″>

              at1=”3at1″
        at2=”3at2″ />

Arrrgh!
It did it again!!! So here is observation number two:

Finally, WeSay uses an XPathNavigator in some places and in the course of my testing I noticed that XmlWriter.WriteNode() behaves slightly different when it is passed an XPathNavigator rather than an XmlReader. Specifically, it seems to always ignore whitespace. So passing an XmlReader (with IgnoreWhitespace = false) to WriteNode for the first file and an XPathDocument for the second file where the files look like this:

File 1:

File 2:

at1=”3at1″ at2=”3at2″ />

Results in a result file looking like this:

    at1=”at1″
    at2=”at2″>

        at1=”3at1″
        at2=”3at2″ />

    at1=”at1″
    at2=”at2″>


        at1=”3at1″
        at2=”3at2″ />

Here’s an outline of the code:

XmlReaderSettings readerSettings = new XmlReaderSettings
                                                   {
                                                       ConformanceLevel = ConformanceLevel.Fragment
                                                   };

XmlWriterSettings writerSettings = new XmlWriterSettings
                                                   {
                                                       ConformanceLevel = ConformanceLevel.Document
                                                   };

XmlReader reader = XmlReader.Create(stream0, readerSettings);
XmlReader reader2 = XmlReader.Create(stream0, readerSettings);
XmlDocument document = new XmlDocument();
document.Load(reader2);
XmlWriter writer = XmlWriter.Create(stream1, writerSettings);

while (!reader.EOF)
{
writer.WriteNode(reader, true);
}
writer.WriteNode(document.CreateNavigator(), true);

Note that this is the case even when you create an XmlDocument from an XmlReader with IgnoreWhistespace = false.

So that about wraps it up. This was not meant to be an exhaustive study of
all the Xml- Reader/Writer/Document/WrietSettings/ReaderSettings/XPathNavigator interactions so if you find anything else unusual or that I grossly misrepresented something please feel free to let me know!

Addendum:
After testing mono’s response to all this upon Cambell’s request I discovered an idiosyncrasy in .net . It seems that attributes with a preceding namespace are not indented so:

File 1:
“xmlns:at1=”at1″” at2=”at2″”>”

Results in:

    at2=”at2″>

              at1=”3at1″
        at2=”3at2″ />


    at1=”at1″
    at2=”at2″>

              at1=”3at1″
        at2=”3at2″ />

Unfortunately (from a cross platform consistency standpoint), Mono does not exhibit this behavior and (correctly?) inserts a newline before that attribute.

Merging LIFT dictionary files

John Hatton — Thu, 21 Jan 2010 07:18:49 +0000

If you aren’t yet using the new collaboration features of WeSay, you may have multiple versions of your dictionary out there. Here are a few notes on ways to get them together.

The simplest case is where the users have been working on completely different sets of words, with no overlap. That is, they each started with completely empty dictionaries, which have never once been merged together. In this specific case, you can merge them by hand. Do that by opening each .lift file and copying all the … chunks of one file in next to the … chunks of the other file. Open in WeSay to make sure you didn’t mess the lift file up.

In the more general case, you will want to merge them together using FieldWorks Language Explorer (FLEx). To do that, follow these steps:

1) Create a new project using FLEx.

2) Import each .lift file into the project, one at a time, until you have a nice combined dictionary.

If getting/installing/using FLEx seems like to much, you can always just ask for someone to do this for you. Write to the WeSay email list and ask someone to do the merge for you.

New Writing System UI

John Hatton — Mon, 04 Jan 2010 07:27:18 +0000

This posting will be of interest only to developers currently using, or considering using our .net Palaso Library, which provides components to do many common language software tasks. This post will look at just one of those, setting up set of Writings Systems.

Well, each year ‘round this time I take a break from my normal obligations and do something interesting, or learn something new. Alas, this year, I did neither. Instead, I squandered the time doing some long “unfinished business”. A couple of years ago, we added to Palaso some pretty nice support for LDML, the standard XML format for writing systems. Writing systems? BORING. I know, I know. Anyhow, the GUI that we had was just the minimum, and not helpful enough to actually ship in our flagship product, WeSay. So WeSay limps along on an older, pre-LDML system. Early in 2009 I did a UI design of what we really need, but, alas, noon actually implemented it. My teammates mentioned they were a bit peeved at me for not letting that bare-bones GUI ship.

adobe acrobat 9 pro
buy adobe acrobat 8
Adobe CS4 Master Collection mac
Buy Corel Draw X4
Adobe Photoshop CS4 Extended
ptc mathcad
buy microsoft office 2007
autodesk autocad 2009
autodesk autocad 64 bit
Microsoft Windows 7 Ultimate
turbo tax 2006

The only interesting thing to me about writing systems, especially user interfaces for them, is that we keep finding it so hard to get them right! I’ve seen half a dozen attempts in the last 10 years, just within the confines of SIL & friends. Here’s why, in my opinion: The vast majority of users and languages have pretty simple needs in this area. The rest, well, they’re pretty complicated (like the dictionary we worked with in South East Asia which has to handle scripts of Thailand, Burma, China, a Romanization, and IPA). Yet all the UI’s we’ve done have catered to both of these equally, and so most people were blocked, they need to call in more geeky help to get past this part of setting up their software. The key insight to fixing this, I think, is that people in typical situation will be shocked by the complexity needed in the non-typical situations. And those who have it hard, well they’ll expect things to be non-trivial. So this latest attempt uses progressive disclosure to keep things simple for most people.

In this post, I’m not going to talk about a lot of the really cool, problem-solving parts of this system which aren’t new. The non UI parts, the ones my colleagues actually give a hoot a about. I don’t think we ever did blog about them, though, so tag-your-it, Cambell

The Writing System Repository Pane

Here’s the equivalent in the latest Palaso system:

There are two main parts to this design. First, the tree on the left organizes your repository in a hopefully easy to understand way, while at the same time giving you shortcuts to what you most likely want to do next. That is, it:

Shows you the writings systems in your repository, grouped by language
Makes suggestions about other writing systems you may wish to add to existing languages (e.g. IPA).
Make suggestions about other languages you may be working on, based on what your Operating System thinks (here, Icelandic and Arabic on my machine).

As a developer, you can control which kinds of suggestions make sense for your application, to keep things simple. For WeSay, for example, we offer voice writing systems, but phonetic transcription and dialects are pretty unlikely. So we’d set up the Suggestor accordingly:

You might have noticed that there are no suggestions under English. Again, this it to keep things simple for the majority of users. We do that by specifying:

Note, even without those suggestions, someone could still make multiple Writing Systems for English easily enough, if they need to.

The Identifiers Tab

The second leg of the design is an Identfiers Tab which stays as simple as possible. As you know, there’s more to life that the good ‘ol “Ethnologue Code” (Now a’ days ISO 639-3). In addition to that, we need to help people come up with a proper RFC5646 identifier, including handling situations common in linguistics which aren’t spelled out by that standard. This is the job of the Identifiers tab.

For the simple (and normal) case, we don’t need to say any more than what the language is. Notice how in the image above, the Identifiers tab has just two controls.

After adding a simple writing system for a language, the next most common thing for users to need is a way to write the language phonetically or phonemically, using the International Phonetic Alphabet. Clicking the provided button just under the Aarri label gives:

Notice in the upper left, this new writing system has been grouped underneath the plain ‘ol Aari one. And on the right, notice that the “Special” combo now says “IPA Transcription”. If we want to specify phonetic vs. phonemic, we can do that with the “Purpose” combo.

If we need a new dialect, clicking the provided button brings up a dialog asking us to type in the name of the dialect. I entered “Foo”, and now we get:

Notice, with the “Special” combo set to “Script/Variant/Region”, we get the more control over the Writing system and its RFC5646 identifier (displayed in the upper right).

Ok, that’s mostly what I wanted to show. When you click “Add Language”, of course, you get searchable list of known languages. And under that More button in the lower left, you get these rarely needed commands:

There’s still some work that could be done (notice Region doesn’t offer a list), but my New Year’s break is over, so that’s it for now. The Palaso Library lives in Mercurial, at http://projects.palaso.org/projects/show/palaso.

Testing the New WeSay Collaboration Features

John Hatton — Mon, 21 Dec 2009 02:24:54 +0000

Since we first introduced WeSay a couple years ago, we’ve heard one request over and over: “Make it so that multiple people can collaborate on the dictionary”. The need for this is actually greater than it was for ShoeBox, ToolBox, Lingualinks, or FieldWorks. These other programs were primarily for linguists, and linguists working on smaller languages rarely have the luxury of a several partner linguists. In contrast, it’s very likely that multiple members of the language community will want to contribute to a WeSay project. In addition, WeSay is designed to be collaboration between and advisor and a native speaker. They will most always have different computers and live in different places. All that to say we’ve known this was important for years, but it’s quite a challenge to get the collaboration powerful enough and yet simple enough for WeSay users. Building this system, known as Chorus, has consumed the lion’s share of our new feature time over the last year.

There’s another reason why we’ve been slow to put this feature out, and why we still (as of December 2009) are only slowly, cautiously introducing it. It turns out that when we add a system which hooks all the team members together, we’ve just dramatically increased the potential for the entire team to suffer from one person’s mishap. For example, imagine that one user deletes or renames a critically important file. If that change is transmitted to the rest of the team, none of them will be able to keep working. Some of these problems have proved hard to predict ahead of time. As we identify such scenarios, we attempt to add protection against them.

We understand that people are tired of waiting. And we could use more feedback to get this ready for Prime Time. So I’m writing this blog for those of you who are willing to spend some time finding out if WeSay’s collaboration features are solid enough for your team. IF you find that they are, then great, you can start benefiting from them. If you find that more work is needed, and you tell us about it, then we’ll be able to better prioritize our efforts to get you what you need.

Note: this blog entry is a work in progress… I expect to improve it over the coming weeks, as people try it out and give me feedback.

Eventually, getting started with WeSay and Language Depot needs to be a lot more simple than it is today. Please don’t hesitate to ask for further help.

Get a LanguageDepot.org Account for the Manager

Go to Language Depot and create yourself an account.

Please don’t use the same password you use for anything important… WeSay is NOT going to be careful about keeping your password well-hidden.

Get a LanguageDepot.org Project for the Language

Write to admin@languagedepot.org. Please provide the following information:

The name of the account you created in the previous step
The name of the project. Normally, the name of the language works well for this.
The ISO 639-3 code for the language. Easiest way to find that is to Google for “ethnologue thelanguagename”.

You will be the initial “manager” of the repository. You will be able to assign contributors to the project, and turn features of the web site on and off.

People you have not added to the project will not be able to access your data. However, we wouldn’t pretend to promise any real “security”. If you need that, it’s perfectly OK to use a Mercurial server somewhere else… you aren’t tied to LanguageDepot.org.

Unless you tell us otherwise, we’ll assume it is ok for us to occasionally look at the files in your repository project for the purpose of fixing a problem for you or seeing how the collaboration features are being used in real projects.

Get LanguageDepot.org Accounts for the contributors

Create accounts for the contributors just like you did for yourself. Then go to the settings for the project and assign each of those people as contributors to the project.

Get the Data Together

It’s important that there is a single, up-to-date copy of the dictionary when you first put it up on Language Depot. If there is currently only a single person working on the dictionary, you need to get their project, and delete the project from their computer. That does two good things: ensures they don’t keep working on it, and ensures that they will be using the proper version of the project later.

Make sure you backup the WeSay project from the stable version of WeSay you’ve been using (e.g. version 0.6). If you decide that the development version (0.7) isn’t ready for you yet, you’ll want to go back to 0.6, and the files are not backwards-compatible.

If there are multiple copies of the dictionary out there, you need to do that for each one of them. That is, get the project, remove it from their computer. You have an extra step in this case, which is to merge the entries together. The simplest case is where the users have been working on completely different sets of words, with no overlap. That is, they each started with completely empty dictionaries, which have never once been merged together. In this specific case, you can merge them by hand. Do that by opening each .lift file and copying all the … chunks of one file in next to the … chunks of the other file. Open in WeSay to make sure you didn’t mess the lift file up.

In the more general case, you will want to merge them together using FieldWorks Language Explorer (FLEx). To do that, follow these steps:

1) Create a new project using FLEx.

2) Import each .lift file into the project, one at a time, until you have a nice combined dictionary.

3) Export the dictionary as a .lift file. That file needs to be placed in the WeSay project you will be pushing up to Language Depot.

Get the Most Recent Version of WeSay

As of December 2009, these features are too new for us to support a large number of people using them. Therefore, you have to go “to the factory” to get it. To get instructions on where to get it, simply email me (hattonjohn on gmail).

Install TortoiseHg

WeSay uses the Mercurial version control system to support sending data around and keeping a history of it. For Windows, the easiest way to get Mercurial is from a free system named “TortoiseHG”, which you can download from here.

Push the project up to LanguageDepot

Ok, once you have a single dictionary folder with the whole team’s data, it’s time to do the initial push up to LanguageDepot. At this point, WeSay can’t do this initial push for you. The easiest way is to open a good ‘ol “command prompt” window. If you don’t know how to do that, probably you’ve already enlisted a more technical person in previous steps. Get them to do this step too.

First, make sure Mercurial (hg) is properly installed. Type “hg version ”. You should see a little version message:

Now, change the directory to be that containing the WeSay dictionary.

Next, push the project up to Language Depot. Here, instead of “wagi”, you’d type the ISO 639-3 language code you used when setting up the project.

You will be asked to enter the user name and password you used in setting up your Language Depot account.

Pull the project down to your computer

I recommend that you now pull the project right back to your computer as if you had never had it. So if the project is already right where you want it, move it, rename it, or back it up and delete it. Now, run the WeSay Configuration Tool, and click “Get From Internet”:

Here comes the most un-cooked part, where we make you type a whole bunch of stuff into one big scary URL:

So, if your account on Language Depot is “lucy_loo” with password “v9isual9”, and the ISO code for the language is “foo”, then you’d enter:

http://lucy_loo:v9isual9@hg-public.languagedepot.org/foo

Give the project a name, and then click “Download”.

Turn on Collaboration

Ok, it’s dumb that this isn’t automatic yet, but you need to enable Send/Receive:

This will cause a button to show up on the dashboard:

Clicking it there (or even while you’re still in the WeSay Configuration tool) will show a dialog like this (sorry about that obvious bug in the text at the bottom):

Clicking “Internet” starts the synchronization:

Note: when WeSay detects that some changes were pulled down from the internet, it closes down and restarts itself so that it has a nice clean start with the new data. We plan to make this unnecessary in some future version.

Pull the project down to the computers of the rest of the team

Now do the same for each member of the team, as you get Language Depot accounts for them. If you have limited internet connectivity, you can speed things up by instead using a USB flash drive.

Like most things in WeSay, you, the Advisor, need to turn on the collaboration features which are appropriate for your project. Some of this could eventually be done automatically, but for now, you need to do this for each user (or do it for one and then copy their “.userConfig” file for everyone else, renaming each copy to match each person’s user name).

“Advanced History”

There are two optional tasks you can enable if you want:

These show up on the dashboard under the “review” section:

The history show you all the changes the team has made:

It’s labeled “advanced history” because, frankly, this is more complex than what we expect many WeSay users to handle. Use your own discretion. It may be that you, the advisor, will want this enabled in your configuration, but not in that of the rest of the team.

The Notes Browser

Collaboration is more than just sharing changes. It turns out that as soon as you start working with others using Send/Receive, you find you want to write notes to them. So we added the ability to attach questions to lexical entries, and to carry on conversations about the question. Future versions will add other kinds of notes, and allow them to be attached to particular fields, not just whole entries.

In the following screenshot, notice the circled buttons. We’d click the left-most to add a new question to this entry. The other button represents a previous, unresolved question. A click on it brings up a dialog box in which we can read and answer the question.

Ok, but how do you find which entries have unresolved notes? WeSay 0.7 also introduces the Notes Browser, which lets you find and interact with notes from all over the dictionary:

In addition to Questions, WeSay currently supports just one other kind of note, the Merge Conflict. These are created by the automatic merger when two team members edit the same field at the same time. Unlike traditional version control systems, Chorus (the engine we’ve written to do all this) doesn’t stop the merge and force you to deal with the problem immediately. Instead, it makes its best guess as to what to do, then creates a Merge Conflict Note which the team can read and deal with when it is ready.

If you don’t like what the merger did, go the entry and make whatever changes are necessary. Then click the “resolved” box to show that this has been dealt with. Or if you need to discuss what to do with a teammate, add a new message to the note. In the following screenshot, I’ve highlighted the hyperlink at the top, and the Resolved box at the bottom.

Ok, sorry, I know that’s a lot to absorb in on sitting. When we’re beyond this alpha testing period, I imagine I’ll reintroduce all of this pieces, with greater detail (though still lacking pedagogical skill :-) ).

WeSay Stable for Linux

cambell — Tue, 25 Aug 2009 03:09:39 +0000

WeSay is now available for Linux in two flavours. The wesay-stable package will install the latest 0.6 stable series, whereas the wesay package will install the latest 0.7 development series.

Full installation instructions are available on projects.palaso.org.

Using SOLID to convert to one-ps-per-sense format

John Hatton — Fri, 26 Jun 2009 23:59:15 +0000

This week saw another colleague here in Papua New Guinea deciding to move his Toolbox dictionary to FLEx. He had a fair amount of entries like this:

\lx ba
\ps n
\sn 1
\ge brother-in-law
\de brother-in-law: reciprocal term between wife’s father’s brother’s children and father’s brother’s daughter’s husband
\sn 2
\ge brother-in-law
\de brother-in-law: reciprocal term between sister’s husband and wife’s siblings
\sn 3
\ge in-law
\de reciprocal term between father’s sister’s husband and wife’s brother’s child
\dt 18/Sep/2000

These are all nouns, and he identifies this just once, at the top of the entry. That counts as good MDF. The problem is, FLEx import doesn’t handle this situation. In fact, a recent import I did left us with over 60 cases where the \ps was turned into its own sense, followed by all the actual senses which were left with no grammatical category. Neither I nor my colleague caught this until she had already been working in WeSay on the new data for too long to go back and repeat the import. Yuck.

To prevent this, we need to move that \ps down to under the \sn, and then copy it for all remaining senses which lack a \ps. As of SOLID version 0.9.319, we can now do this:

As with all these quickfixes, I use TortoiseHG (mercurial) so that I can look at exactly what changed, verifying that nothing was messed up. Here’s what quickfix did to the above record, as seen from TortoiseHg’s Commit tool:

More control over “missing info” tasks

John Hatton — Mon, 22 Jun 2009 05:32:04 +0000

WeSay has always had Tasks which would show you just the words that needed some more information in a particular field. However, the selection of which entries to show was pretty blunt: if the field had an empty slot in any of its multiple writing systems, the task would show that entry. This meant that you couldn’t easily set up WeSay for a user who, for example, just wanted to add vernacular definitions where English ones had already been entered.

In another case, we might want to set a user up to add voice recordings of example sentences. But the task should only show example sentences where someone had previously entered in the example text.

The latest development release (0.5 build 2000) addresses this. When you first create a project, tasks are configured to have the same behavior as before: an entry will be chosen if *any* of the writing systems assigned to that field are empty.

You can now limit the task to filling in the vernacular (gaw, in this example):

In addition, we can limit the task to only those entries where some other writing system has already been filled in:

Thanks, Mark, for taking the time to submit this request. We would appreciate any feedback you can give us on this feature. Does it work well for you?

The obvious next step would be to add a way make duplicates of some tasks, so that you could have both an “Add Examples” and an “Add Example Recordings” task. This is now possible by editing the .wesayconfig file in a text editor. If you want to know how, set me an message (hattonjohn at gmail). That will tell me how much demand there is for it, and if there’s enough, we’ll make it easier.

WeSay, Palaso, and Linux

cambell — Fri, 29 May 2009 09:15:27 +0000

Feeling in somewhat of a reflective mood I thought I’d write about the current state of our work with Linux, and have a look at where we hope to go in the next few months.

Over the last few months (er year) we’ve been working on getting our applications to run on the Ubuntu Linux platform. We are hoping that this will enable our applications to be well used on the new breed of net books and other lesser specified machines. The main work has been centered around the mono project which enables .Net Winforms applications (most commonly a Windows technology) to run on Linux. So, where are we now?

The http://packages.palaso.org repository has been launched to make our software available to you all.
WeSay 0.4 (stable) is available from packages.palaso.org. Install details are on the WeSay Wiki on projects.palaso.org. This is an early release and has a number of issues that we know about. Have a look at the WeSay bug tracker for further information.
OurWord is available from packages.palaso.org. Install details are on the OurWord Wiki on projects.palaso.org. Again this is an early release, there are a number of visual issues which are being tracked on the OurWord bug tracker.
Automatic USB stick detection now works. That means in WeSay, you can click backup and expect it to wait for you to put a USB stick into your computer and the backup will ‘just work’.

There are a few significant issues that we’ve noticed and we’re working on fixing these up over the next month or two.

Keyboard switching doesn’t work. In WeSay we make extensive use of keyboard switching. If you’re editing a field that’s say IPA they keyboard should be set to IPA for you. When you tab to another field with a different writing system the keyboard should change automatically. We are fixing this by contributing to the SCIM project. We’re extending it’s capability to support our Keyboard Switching component. We expect to have this available very soon.
Non Roman Script rendering using Mono doesn’t work. We’re fixing this in Mono, by re-writing large chunks of the Text Control. This will take some time to filter through and become available - perhaps some time in the next three months.
The WeSay development branch (0.5.x) doesn’t currently work with Mono. We’re working on bringing that up to date so that we can release on Windows and Ubuntu simultaneously.

Once we’ve finished these issues, we’ll be back to focusing on core features of our products rather than enabling Linux to run them. At that time we’ll leave the improvement of Mono and Linux to others.

Well, enough reflecting. Time to go get a coffee and carry on with the work.

Further details can be found on our various projects at http://projects.palaso.org

More Quick Fixing For SOLID

John Hatton — Thu, 16 Apr 2009 07:38:11 +0000

Yes, I’m WALLOWING in a swamp of Standard Format files this week. Soon we’re doing a Lexicography workshop here in PNG, and in preparation, several of the participants are wanting to switch their 20 year-old dictionaries to FieldWorks Language Explorer. I’ve offered to get their old SFM dictionaries in, and, Boy am I glad we have SOLID. Since we find it hard to prioritize work on SOLID over our other projects, like WeSay, I decided I should stay sane through this awful job by hacking on SOLID a bit as I go. I wrote about my first steps yesterday. Today, with version 0.9.302, we have a few more nice things:

Improved QuickFix: MoveUp capability to handle multiple root targets, not just \lx
Do alphabetic sort on markers when rebuilding the list.
Add “remove empty” quickfix
Ask user about saving when they exit.
Show marker that is the focus of the filter in bold, to make them easier to find.

Here’s a screenshot of the QuickFix dialog at the moment… I know it’s ugly, but I’m aiming at making my life easier here…

And

This last one might make more sense if you only specified which markers to *not* delete. What would they be? \sn? \rf? David Coward, do you read this blog?

You may notice, there’s little rhyme or reason to the default options here… it would be a nice contribution if someone in the community would compile a good set for us. Send ‘um in, and we’ll add it to the program.

New Quick Fix for SOLID

John Hatton — Wed, 15 Apr 2009 05:50:03 +0000

One of the most painful parts of cleaning up a dictionary is this kind of situation:

\lx foo

\ps n

\ge blah

\bw German

\gr wanem

Here, the problem is the \bw(borrowed word) is an entry field, but it’s sitting in the middle of the sense fields.

SOLID 0.9.297 adds a minimal “quick fix” which can grab these kinds of fields and pull them up to the top of the entry. You specify the marker, and the quick fix does it for all the entries. As the quick fix dialog box says, this is dangerous, and I personally use it only because I also use Mecurial to take snapshots of my changes to files in between actions like this, in case something goes wrong. But that’s a whole ‘nother post…

Other changes:

You can now click on the “count” table header to sort by marker frequency.

Uses the new Palaso system for defining writing systems.

Removed the mapping to FLEx, since nowadays, there’s no point. If you want to use FLEx, either directly import the cleaned-up MDF (preferred), or export a LIFT file and import that into FLEx.