A quick trip to LiveDocs for EUR-Lex bilingual texts

Quite a number of friends and respected colleagues use EUR-Lex as a reference source for EU legislation. Being generally sensible people, some of them have backed away from the overfull slopbucket of bulk DGT data and built more selective corpora of the legislation which they actually need for their work.

However, the issue of how to get the data into a usable form with a minimum of effort has caused no little trouble at times. The various texts can be copied out or downloaded in the languages of interest and aligned, but depending on the quality of the alignment tool, the results are often unsatisfactory. I've been told that AlignFactory does a better job than most, but then the question of how best to deal with the HTML bitexts from AlignFactory remains.

memoQ LiveDocs is of course rather helpful for quick and sometimes dirty alignment, but if the synchronization of the texts is too many segments off, it is sometimes difficult to find the information one needs even when the (bilingual) document is opened from the context menu in a concordance window.

EUR-Lex offers bi- or tri-lingual views of most documents in a web page. The alignments are often imperfect, but the synchronization is usually off by only one or two segments, so finding the right text in a document's context is not terribly difficult. So these often imperfect alignments are usually quite adequate for use as references in a memoQ LiveDocs corpus. Here is a procedure one might follow to get the EUR-Lex data there.

The bilingual text of a view such as the one above can be selected by dragging the cursor to select the first part of the information, then scrolling to the bottom of the window and Shift+clicking to select all the text in both columns:

Copy this text, then paste it into Excel:

Then import the Excel file as a file for "translation" in a memoQ project with the right language settings. Because of quirks with data access in LiveDocs if the target language variants are specified and possibly not matched, I have created a "data conversion project" with generic language settings (DE + EN in my case as opposed to my usual DE-DE + EN-US project settings) to ensure that data stored in LiveDocs will be accessed without trouble from any project. (This irritating issue of language variants in LiveDocs was introduced a few version ago by Kilgray in an attempt to placate some large agencies, but it has caused enormous headaches for professional translators who work with multiple sublanguage settings. We hope that urgent attention will be given to this problem soon, and until then, keep your LiveDocs language data settings generic to ensure trouble-free data access!)

When the Excel file is added to the Translations file list, there are two important changes to make in the import options. First, the filter must be changed from Microsoft Excel to "multilingual delimited text" (which also handles multilingual Excel files!). Second, the filter configuration must be "changed" to specify which data is in the columns of interest.

The screenshot above shows the import settings that were appropriate for the data I copied from EUR-Lex. Your settings will likely differ, but in each case the values need to be checked or set in the fields near the arrows ("Source language" particularly at the top and the three dropdown menus by the second arrow below).

Once the data are imported, some adjustments can be made by splitting or joining segments, but I don't think the effort is generally worth it, because in the cases I have seen, data are not far out of sync if they are mismatched, and the synchronization is usually corrected after a short interval.

In the Translations list of the Project home, the bilingual text can be selected and added to a LiveDocs corpus using the menus or ribbons.

The screenshot below shows the worst location of badly synchronized data in the text I copied here:

This minor dislocation does not pose a significant barrier to finding the information I might need to read and understand when using this judgment as a reference. The document context is available from the context menu in the memoQ Concordance as well as the context menu of the entry appearing in the Translation results pane.

A similar data migration procedure can be implemented for most bilingual tables in HTML files, word processing files or other data sources by copying the data into Excel and using the multilingual delimited text filter.

The state of the upgrade: memoQ 2013 R2

The memoQ 2013 release started off on the wrong foot with me in many ways. I was deeply disappointed by the features that were previewed in Budapest at the last memoQfest, and I was even less happy after I saw what a hash had been made of one of the features I use most: comments. In fact, I wrote a rather annoyed blog post about that not long after the release. There was a lot of talk about "game-changing innovation", but frankly I really could not see it. My translating colleagues asked me if it was worth it to upgrade, and aside from my usual warnings about the need to wait for at least 2 or 3 months after any release for it to mature and stabilize, I just could not find any compelling arguments for a freelance translator to move from the stable, excellent 6.2 version to the rather dodgy 6.5 version, or "memoQ 2013" as it was rechristened.

Almost on the usual schedule, however, two months later the bugs were largely sorted out, the initial mistakes in the comment feature redesign were well fixed, and I no longer saw the memoQ 2013 release in the same dim light, but could actually see some benefits for my freelance colleagues to upgrade to that version and no actual harm in doing so. And as I got to know the fuzzy term matching feature better and saw how it helped me deal with typo-laden source documents or the usual spelling chaos of German technical writers, I began to see some very compelling value in memoQ 2013 for translators.

Most of the "game changers" talked about in May actually arrived a month ago with Release 2 of memoQ 2013. I did my best to lower expectations for this release, not because I think it is crap, but because I think this is one of the best CAT tool version upgrades I have seen in 13 years, and I knew it would need the usual time to mature. I think by the end of the year this version will have so much to offer that I would rather not have people stressing over the small stuff that I am confident will be fixed well.

However, I decided to live dangerously, and I switched over all my production work to use this version even before the official release. The first few weeks were not fun with all the little quirks I discovered and duly reported, but I encountered nothing data-destroying or really shocking, mostly just housekeeping details like somebody forgetting to vacuum the rug after gutting the whole house and giving it a nice remodel.

One month after the official release, memoQ 2013 R2 is far more reliable than I remember any memoQ version being one month after release. There has been steady refinement in its features, and I continue to discover hidden gems that I sometimes suspect most of the Kilgray team aren't even aware of yet because so much was added and changed, but not in a way that disrupted older work processes. I have a long shopping list of refinements that I think should be made to new features like the TM search tool (which has only actually worked on my system since the release of the 6.8.5 build about a week ago) or that ground-breaking monolingual review feature which (will probably be the next big CAT feature to copy), but even the new features I consider rather immature are already looking pretty damned good. I can't guarantee that this release can be trusted for all your work right now (though it actually seems pretty good to me right now), but since it can be safely installed in parallel with older versions, I definitely recommend taking a look and joining the conversation on refinements still needed. I think Kilgray has been very responsive to user feedback in this round, and I can't say I am anything but encouraged by what I have seen in the last month.

One very exciting change for me in the current build (6.8.6) is that the rather risky non-optional export of target text comments with DOCX files has been sorted out very nicely. The solution seems a little strange to me right now, but it's a great step forward with some excellent possibilities.

When I saw those "severity levels" added to the commenting features in memoQ 2013 (6.5), I had very little good to say about them. I still don't think much about how they are named and wish I could choose my own labels, but now I can only applaud their usefulness. Why? Because the addition of the five checkboxes above has given me the control I want over comments to be included in an exported translation of a DOCX file. I can cleanly separate the comments which are notes to myself from those for my project partners and comments for my customers. This is very helpful.

I do think it is odd that this control was placed at Tools > Options > Miscellaneous > Translation when the comment exports (as far as I know) only affect DOCX files, but if there are plans to extend this feature to other exported formats, then this makes sense. I would like to see similar filtering controls for the ordinary view filters (on that last tab where comment and tag filtering criteria can be specified) and for comment inclusion in a bilingual RTF export. Either of these would be an enormous help to my frequent work processes, because I use a lot of comments intended for different people, and sorting these out cleanly can be laborious.

In recent weeks I have been working on the new edition of my memoQ tips book and taking a very close look at "corners" of the software that I suspect very few have time or inclination to look in. And I've had days when it really felt like Christmas has come early. One discovery after another of nice little refinements, lots of incremental improvements, which added together give a total with what I feel is a lot of value. I'm writing way too many private thank-yous to some of the people at Kilgray for what I see as excellent new directions even if I am inclined to argue over some of the details.

Since the release of memoQ 6.2 and its follow-ups with the bilingual text/Excel filter, there has been such a steady flow of useful improvements to help individual translators work better that those who claim that all the effort of development has been spent catering to the corporate sausage-making interests of the low-paying cattle call crowd simply haven't been paying attention. Or they have been confused by Kilgray's occasionally appalling failure to organize their messages properly for different interest groups. If you're talking to a big group of freelance translators and start discussing "great server features to monitor your translators' productivity", don't expect blown kisses and showers of rose petals. Sometimes it's obvious that the makers of the tool don't always understand the importance of what they have created for our work. Well, why should they? We're the ones doing it. But I tell you, right now there is a lot more gold for individual translators in the memoQ mine than anyone realizes. That goes for me too. I am surprised by fat new nuggets I find almost every week.

Do I care that so much effort is spent on developing cutting edge project management features for memoQ translation servers, even ones that I think can be abused in some pretty awful ways by some companies whose business practices I detest? Well yes I do... I think it's great. Besides, I can actually come up with nice uses of those awful features. You can do a lot of things with a cutting-edge: chop up a tasty salad... or the local nursery school. Blame the fool, not the tool.

Kilgray has avoided the disastrous errors committed by Atril in the last decade as their market mis-focus and disastrous failure to get the maintenance revenue needed to fix and develop features steadily eroded the ability of its loyal users to cope with a changing market. There was nearly a complete failure to compete for the business of translation agencies and corporate and government translation departments. And the solutions that prevailed in those quarters were mostly rather awful. I watched whole departments of Siemens traumatized by the disastrous Trados Teamworks, which made a number of those in the translation team of the medical products division look forward to retirement.

Kilgray has steadily built its business in the markets ignored by Atril a decade ago and in doing so has secured its future far better and ensured the funding of a truly remarkable series of improvements in the four and a half years I have been using memoQ. And now... when I look at the features of the recent SDL Trados 2014 release I see good things that I have known from other tools for a long time for the most part, nice to have really, but as I stifle a yawn I wonder if it all really has to be so complex since I'm not depending on consulting or training for SDL to pay my bills. And then I get back to memoQ and keep getting rocked by the "wow factor" as I find useful new things while trying to concentrate and get a job done. memoQ 2013 R2 is one of the worst offenders I've seen in a long time for its very real threats to make my work a lot easier and more fun!

A new look for MultiTerm XML data in memoQ & Trados

Recently I got back to testing suggestions made last year for improving the quality and usability of terminology data from memoQ and Trados MultiTerm. With a bit of a refresher for rusty XSLT skills and brilliant help with sorting challenges from Stefan Gentz of Tracom, things are now looking quite promising. Here's a first look at early results for HTML conversions of term data in MultiTerm XML format:

This approach, perhaps including other useful conversions, will be included in the chapter on "possibly useful scripts and macros" in the tutorial guide memoQ 6 in Quick Steps, which will be released very soon with over 200 pages of productivity suggestions for the latest version of Kilgray's desktop technology.

memoQ 6 desktop: working with other memoQ users

The best methods for memoQ desktop editions to work with other memoQ users are influenced by the versions of the software you and others use. If these versions are compatible, project information can be shared fully, including previews and status settings for the translation. Otherwise, many of the compromises of working with other translation environments apply.

Bilingual exchange files

Bilingual exchange files are generated via Project home > Translations > Export bilingual. If the other person also uses memoQ 6, the best and "friendliest" option is to create a memoQ XLIFF and include the skeleton and preview. The "skeleton" allows target files to be created. For earlier versions of memoQ, a simple XLIFF with the extension changed to XLF or one of the other bilingual formats will do. In memoQ 6, bilinguals are imported using the Import command (and are recognized automatically); in earlier versions, the Import/update bilingual command is used. The tags will always be respected.

If the bilingual DOC format is used for exchange, the finished work must be exported via Export bilingual. Other export commands produce monolingual target documents. The least complicated format to use for users of memo 4.2 to 5.0 is the two-column RTF.

Version 6 TMs and termbases are fully compatible with Version 5 of memoQ, and data can be exchanged with all versions via TMX and delimited formats.

Project backups

A fully configured project with all settings, translation files, TMs, termbases and corpora can be sent by creating a backup. On the memoQ Dashboard, select the project and click Backup selected. Warning: backup files can be very large, so you might want to detach very big TMs first, for example. And of course this requires the same version of memoQ to be used.

Handoff packages (PM version only)

If translations have been assigned by name in Project home > Translations, handoff packages for translators and reviewers, including necessary resources, can be created after running a check on Project home > Overview > General > Handoff checks.This, too, requires the same version of memoQ.

Another point to consider: memoQ versions 5 and 6 can co-exist on the same desktop computer, so if you need to continue working with clients who have the version 5 server, for example, there is little to stand in the way of upgrading to version 6. The only real difficulty might arise if you want to attach corpora you have migrated; this may require restoring the LiveDocs corpora from the backup of the old version or creating a new one.

Reimporting memoQ 6 bilingual files

Changes in the translation menu options in version 6 of memoQ have caused a little confusion with some users. The differences in handling "memoQ bilingual" files, which are used for external translation or review, seem to cause particular stress in some cases.

In previous versions of memoQ, the bilingual files had a special command for importing, re-importing or updating (Import/update bilingual). In memoQ 6, this was consolidated with the Import function, which replaces its predecessor in earlier versions, Add document.

memoQ automatically detects that the file is a bilingual document associated with a file in the project. When the file is selected, the appropriate import options for the bilingual update will be set automatically:

The file name displayed is actually the name of the file to be updated, not the name of the bilingual file. Clocking OK updates the translation file with the changes in the bilingual document - in the example shown above a bilingual RTF table created from the document to be translated. The results of the update are shown in a dialog:

After that, the changes can be viewed with tracked changes features if desired (if versioning is activated):

Even without tracked changes, the status of updated segments will be set to edited, so these segments can be found quickly using the filters to confirm that the changes are acceptable before the finished document is exported.

Translating "foreign" bilingual tables in memoQ

--- In memoQ@yahoogroups.com, Liset Nyland wrote:
> A client has sent me a 2-column rtf-file export from DVX.
> It looks similar to the MemoQ export but not quite.
>
> The target column is full of fuzzy matches, so I need to recover these.
...
> ... do you know if there's a bilingual format exported from DVX that can
> be loaded and translated directly in MemoQ?

There is one way to deal more-or-less directly with the DVX bilingual RTF tables - or any others being introduced by other providers or bilingual tables that some customers are fond of using to store translation strings or other content. I would love to see a general import routine from Kilgray that allows selection of source and target columns of various file types in a dialog, but until then...

1. Get a copy of the PlusToyZ macros by German/English to Ukrainian/Russian translator Arkady Vysotsky.
2. Copy the source and column targets into a separate RTF or MS Word file.
3. Run the PlusToyZ macro to convert that to a Trados-like bilingual (the old Wordfast/Trados RTF/DOC bilingual)
4. Import the converted file to memoQ using the default filter, which is intelligent enough to recognize that you are dealing with Trados-compatible bilingual DOC/RTF.
5. Translate, edit, feed the TM, etc.
6. Export the processed file.
7. Use the appropriate conversion macro in PlusToyZ to turn the data back into a table.
8. Paste the data back into the original bilingual table from DVX or whatever tool it came from.

This is the preferred method to use when your bilingual table is partially pretranslated, or you have a translated table you want to edit while having a better look at the source text. This would also be a useful method for jobs I've had where customers have string or terminology lists in Excel to translate that are in some cases incomplete.

Once you get to Step 3, you can translate that bilingual format in any tool which works with the old Trados RTF/Word segmentation, such as WordFast Classic.I think that was actually the reason Arkady wrote those macros in the first place.

If you want to protect the DVX codes (or similar structures, including placeholders) or store them in the TM as proper tags, run the Regex tagger or use a cascading filter a described in my other blog post about regular expressions for DVX external table translation in memoQ. Of course, for content other than DVX tags, a different regular expression will be needed.

RegEx for translating DVX external view tables in memoQ

Atril's Dejà Vu was the first translation environment tool I am aware of to offer a means of exchanging translation content for review, correction and translation using an ordinary word processor. These "external views" were the original inspiration for memoQ's RTF bilingual tables, which are used in many interoperable workflows not only with people using a word processor but with many other CAT tools as well.

As with memoQ RTF bilinguals, the content in the "external view" which is not to be translated can be selected and hidden with a word processor, leaving only a target column into which the source text has been copied. But these steps alone with the standard RTF filter pose a problem:

The DVX "codes" (tags), which are represented by curly brackets enclosing a number, are not protected. Erasing parts of them can damage the content. It is also not possible to perform a tag check using the memoQ QA functions.

The solution is to use the Regex tagger in memoQ. There are two ways to do this.

If the document has already been imported,

the tagger can be run from the Format menu.

Enter the appropriate regular expression to convert the DVX code to a protected tag: \{(\d+)\}

This expression describes the pattern of the text to protect: a curly bracket (with a backslash in front of it to indicate that this is to be interpreted literally as a character, not as a bracket for grouping something), one or more digits (\d indicates a digit as opposed to d, which is just the letter d, and the plus sign means one or more) and a closing curly bracket ("escaped" with a backslash so it is understood literally as the bracket character in the DVX code.)

Click Add to put the rule in the list, then click Run tagger now.

The result is protected tags in the translation grid of memoQ. These can also be verified with a QA tag check after the translation is completed.

Your regular expression rules can be saved in the dialog above and re-used, or exported from the list under Tools > Resource console... > Filter configurations and shared with others.

The regular expression tagger can also be used as a cascading filter when the RTF file for the external view is imported:

Here the configuration can also be saved or another one loaded.

memoQuickie: feedback comments

Exporting comments in memoQ in a bilingual table for others to comment further and correct, then re-importing to update translations and the TM is a very useful feature. I use it often to inform clients of uncertain terms or errors in the source text. You can too, as follows:

1. While translating, use Ctrl+M or double-click the speech balloon on the right to open the comment window and comment the segment.
2. On the Translations page of the Project home tab, select the document(s) with comments to export and click Create view. Choose the advanced options:
   
3. Then mark the checkbox for Commented on the Status tab:
    
4. To export the new view as an RTF table, select it and click Export bilingual and choose the Two-column RTF option. To refine your options, click Next, or just export the table.
    

The exported table shows only commented segments: