A Hebrew abbreviations "hint base" points the way for other languages

Years ago I published a guideline for how to create something like a term base for memoQ that can handle the irregularities one might find in the way German attorneys on tight deadlines might type the many abbreviations they use in crazy ways. The memoQ term base model can't cope with punctuation and many special characters, so it's basically impossible to use it to map something like "US-$" to the standard currency code "USD". But regular expressions in an auto-translation rule can do that, of course.

The same principle can be used simply to map abbreviations to their full expression so the translator can decode the abbreviation and decide how to render it. Here's an example of that in Hebrew:

This can, of course, be done in other languages, but the fellow who had this idea and asked me about it happens to be a Hebrew translator working into several target languages. I'm tempted to adapt one of my German abbreviation sets to map to the full German expression in the target to serve as an aid to translators who might not be as familiar with the abbreviations as I am and who are also not bound strictly to a particular target language expression. A cheat sheet, basically, or a "hint base" if there is such a thing.

The code for this is particularly simple. Here's a quick look at the resource in an external editor:

The basic "engine" is just a list (#abbreviations#). And the resource was created quickly using search and replace on a list of over 600 abbreviations in an Excel spreadsheet.

In the awful memoQ rule editor it looks like this:

Those who know Hebrew may note that some periods are out of place. I'm not an RTL expert, so I had a few issues with punctuation migrating as I moved data from one format to another, but someone familiar with issues like that can fix things without much ado. This was just a quick prototype to demonstrate feasibility. And a few minutes of search and replace work in a text editor beats entering more than 600 pairs manually in the built-in editor for memoQ. It would be nice if that damned editor included list import features that would read Excel files directly!

As with other auto-translation rules, certain characters may need to be represented by entities or uuencoding. The simple rule shown above can also be made more robust by dealing with variable punctuation, for example. Complexity can always be added. 

Many thanks to the translator colleague who shared his challenge and gave me something fun to do after a grueling day of mapping many messed-up date formats from a lot of different source languages I mostly don't know :-)

Building a regex-savvy "termbase" in memoQ

For years I have been frustrated by and dissatisfied with how abbreviations are handled in the current memoQ termbase model. The crux of the problem is the handling of the periods in the expressions. This can be seen with termbase entries like the following, for example:

If the abbreviation "Art." appears in the source text, only the second source entry - the one without the period - will give a match result in memoQ. The first entry is simply ignored.

An additional problem which one would face, even if the terminal period character in the term did not pose a problem, is that authors are often notoriously variable in the way they write abbreviations. Take, for example, the abbreviation for the German expression "in Verbindung mit", usually written as "i.V.m."

In recent legal translation work, I have encountered this expression written as above, but also as "i. V. m." (with spaces), "iVm" (no spaces no periods) and sloppily typed variations like "iV.m" or "i. V.m." What's a poor wordworker to do?

The answer came to me while refining a set of auto-translation rules for bibliography formatting and legal references. These, too, can suffer from similar troubles: "page 7" might be abbreviated as "p. 7", but in the sloppy chaos of source texts poorly edited one might find "p.7", "p 7", "p7" or even variations with the letter capitalized, like "P.7". If you are translating nearly 1000 references in a bibliography, robust shortcuts are very helpful and save a lot of time, and if those shortcuts are based on memoQ auto-translation rules, they can also be used in a QA profile to ensure that every bit matches correctly.

As the screen capture from a memoQ Facebook group above suggests, the way to go about this is to identify which parts of the expression might vary with different deliberate and accidental typing. These are usually spaces and periods in the case of abbreviations; sometimes, particularly with German legal abbreviations, capitalization and dashes may play roles as well. (I tore my hair out not long ago trying to understand an Austrian legal text referring to two laws, which differed in their three-letter abbreviations only by a dash inserted after the first letter of one.)

In regular expressions, the question mark character means "zero or one" of whatever character precedes the question mark. So if I want a rule that acts in the case of one or no periods, I put a question mark after the period character. And because in the language of regular expressions, a period is shorthand for any character, if I want to talk about an actual period ("."), I have to precede that character by a backslash ("\."). In the technical jargon of Nerdworld that is known as "escaping the period" and there is no escaping such syntax if you want a regular expression rule about periods, period.

Spaces (normal or non-breaking ones) are represented by an escaped lowercase "s": "\s". So a matching rule for the English abbreviation "e.g" which catches a lot of typing variations might be

e\.?\s?g\.?

And in German, the target replacement rule might be

d.h.

Of course, if a typist is sloppy, there might be more than one space, or a comma might be typed accidentally instead of a period (the keys are adjacent, and if your screen is as dirty as mine gets sometimes, your eyes might not notice); capitalization might also differ accidentally or based on context. The regular expressions for matching can be adapted to handle all these cases if need be.

Rules of this type are not particularly difficult to construct, but refining them to accommodate all the variations you are likely to encounter may require an expert hand. Thus, as I have suggested before,. the average user should focus on documenting all the possible source variations clearly in a table which includes the desired target equivalents, and this table should be given to an expert (Kilgray support, a qualified consultant like Marek Pawelec or a technical programmer familiar with regular expressions and their use in memoQ). Trust me, this will save a lot of frayed nerves and probably significant time and money as well.

So now I am building a few memoQ auto-translation rulesets which are essentially fault-tolerant abbreviation glossaries. These, together with the similar rulesets for formatting bibliographical references and references to sections, paragraphs, lines, margin notes, etc. in laws, have been very helpful in reducing the time spent translating messy legal source texts, and the accuracy of the work has been improved significantly. Give it a try for your translation challenges!

memoQuickie: fixing source segmentation from abbreviations

Do you see segmentation like the above in your projects? Annoying, right? This is easy to fix in memoQ.

Go to Tools > Options... > Default resources > Segmentation rules (in the row of icons):

Select the language (including sublanguage if relevant) and select the editable rule set, then click Edit.

On the tab for custom lists, add the offending abbreviations to the #abbr_short# list.

Re-import the document(s) on the Translations > Documents list of the Project home tab. The number of segments in my document was reduced from 197 to 134, because it was so laden with academic titles. Since I use versioning, any previously translated segments can be recovered quickly by Operations > X-Translate...

Sometimes I think that abbreviations I added aren't fixing the segmentation. In those cases I have usually switched to a sublanguage for which they were not entered.