Twitterview: SDL Trados Studio, memoQ, DVX2 and PDF extraction

When I began using Twitter somewhat hesitantly three years ago, I never expected that it would eventually prove to be one of the most useful social media tools for gathering information of professional value. Much of this is serendipitous; I really never know what will come floating down the twitstream or where some of the conversations in it will go. Like the direct chat I had with with a colleague in New Zealand about features she liked best in the two main CAT tools she uses, SDL Trados Studio and memoQ.

We both really appreciate the TM-driven segmentation in memoQ and the superior leverage this offers. But to my surprise, she expressed a preference for SDL Trados Studio, particularly for the quality of its PDF text extractions from electronically generated files. This is not a feature I make heavy use of in either tool, though I have used it more often lately in memoQ for alignments in the LiveDocs module and found it generally satisfactory. Most of my work involving PDF files is with scanned documents - there one has no choice but to use a good OCR tool like OmniPage or ABBYY FineReader.

So I was quite intrigued that the quality of PDF was "better" than from standalone tools. Especially because my experience is quite different. Further discussion (not shown in the graphic) revealed that what she actually meant was that the quality of the text extraction with the CAT tool usually beat the quality of text received from translation agencies who performed conversions. That is easy to explain, really. In my experience, most agencies are clueless about how to use conversion tools and too often use automated settings and save the results "with layout". This is very often utterly unsuited for work with translation environment tools or requires a lot of cleanup and code zapping.

For years I have recommended to agencies and colleagues that they spare themselves a lot of headaches by saving PDF conversions as plain text and adding any desired formatting later. Most people ignore that advice and suffer accordingly. So in a way, a CAT tool that does so encourages "best practice" for PDF translation for those files they are actually able to handle.

Encouraged by the Twitter exchange, I decided to do a few tests with files from recent projects. I took a PDF I had with various IFRS-related texts from EU publications. It appeared to extract quickly and cleanly in memoQ, giving me a translation grid full of nicely segmented text. SDL Trados Studio 2009 choked badly on it and extracted nothing. Her extraction in SDL Trados Studio 2011 caused a timeout with the project I was told, but the text itself was completely extracted and converted to DOCX format. This is useful, because unlike the extraction to plain text in memoQ, this offers the possibility to add or change some text formatting in the translation grid. Other extraction examples from SDL Trados Studio 2011 showed that text formatting was preserved.

A closer examination of the extracted texts revealed some problems with both the memoQ and Trados Studio extractions. The memoQ 5 PDF text extraction engine proved incapable of handling text in multiple columns properly. The paragraph order was all fouled up. The extraction with SDL Trados Studio had a great number of superfluous spaces. Whether it is possible to optimize this in the settings somehow I do not know. The results of all the extraction tests are downloadable here in a 6 MB ZIP file. I've included the SDL Trados Studio extraction saved to plain text as well for a better comparison of the text order and surplus spaces problems.

Overall, I am personally not very pleased with the results of the text extractions from PDF in either tool. The results from SDL Trados Studio are clearly better, and other examples that were shared made it clear that this tool works better than many an untrained PM with better PDF conversion software. This is certainly much better than solutions I see many translators using. But really, nothing beats good OCR software, an understanding of how to use it well and a proper workflow to get a good TM and target file better fit for most purposes.

*****

Update 2012-05-22: I met colleague Victor Dewsbery at a recent gathering in Berlin, and he told me about his tests with the recently introduced PDF import feature of Atril's Déjà Vu X2 translation environment. He kindly offered to share his results (available for download here) and wrote:

Here is the result of the PDF>DVX2>RTF>ZIP process for your monster EU PDF file. Comments on the process and the result:

• The steps involved were: 1. import the file into DVX2 as a PDF file; 2. mark all segments and copy source to target; 3. export the file as if it were a translated file (it comes out as an RTF file). The RTF file is 20 MB in size and zips to 3 MB.
• Steps 1 and 3 took a long time, and DVX2 claimed to be not responding. For step 1 I just left it and it eventually came up with the goods. Step 3 exported the RTF file perfectly, even though DVX2 claimed that the export had not finished. I was able to open the RTF file (it was locked, but I simply renamed it), and this is the version which I enclose. Half an hour later DVX2 had still not ended the export process (and had to be closed via the Task Manager), although the exported file was in fact perfectly OK. The procedure worked more smoothly with a couple of smaller PDF files. Atril is working on streamlining the process and ironing out the glitches in the process, especially the “not responding” messages.
• The result actually looks very good to me. There are hardly any codes in the DVX2 project file (the import routine also integrates CodeZapper). I didn’t spot any mistakes in the sequence of the text. Indented sections with numbering seem to be formatted properly - i.e. with tabs and without any multiple spaces.
• The top and bottom page boundaries in the exported file are too wide, so most pages run over and the document has over 900 pages instead of just under 500. Marking the whole document and dragging the header/footer spaces in Word seems to fix this fairly quickly.
• I note that some headlines are made up of individual letters with spaces between them. This may be related to the German habit of using letter spacing (“Sperrschrift”) for emphasis as an alternative to bold type.
• I found one instance where text was chopped up into a table on page 857 of the file.
• There are occasional arbitrary jumps in type size and right/left page boundaries between sections.

On the strength of this sample, it would usually be OK to simply import the PDF file into DVX2, translate in the normal way, and then fix any formatting problems in the exported file.

Five favorite things about the CAT

When I listen to colleagues talk about the tools they use, I find it interesting how diverse the points are that they emphasize when describing the advantages of their environments. I'm not surprised, really, because the needs of individuals vary a lot as do the projects they may encounter in different phases of their careers. And the limits of a tool itself influence the "advantages", of course.

So I sent off a few e-mails to some friends, asking what they felt the "top five" features of their choice of CAT tools are for their purposes. I'm still waiting for a few responses, but I'd like to share what I've heard so far. I'll add more as and when I get more feedback. Others are welcome to add their Top 5s in the comments.

OmegaT
The respondent here uses this tool exclusively and does a fine job of correcting me every time I put my foot in my mouth with a misstatement about the capabilities of the software. He wrote with his typical humility:

I don't think I'm the right person to do this. Mainly because OmegaT is the only tool I've used for the last nine years.

Favourite features:

• It runs on Linux.
• It does what it says on the box.
• It *always* works. It doesn't hang or crash. I can't remember when it last did, if it ever did. Bugs are rare and when they do happen tend to concern secondary features that are still at the beta stage.
• It's fast; almost everything is near-instantaneous. (An exception is the "Search files" function, though I don't use that much and I doubt many others do, either.)
• "Upgrading" means downloading and unpacking, that's it.

Not very sexy, sorry. Like my '99 Mazda 626 wagon, it just gets the job done

Just gets the job done? Well, what more could one want? There are plenty of tools that often don't manage that. I've seen huge improvements in this environment in recent years, and there are certainly worse tools to start with. It is Open Source but worth taking seriously for professional work.

SDL Trados Studio
The respondent here is still using Studio 2009, so surely more notable features will be discovered after an upgrade to Studio 2011. But for his team of top-notch translators, the three he indicated are pretty persuasive:

• AutoSuggest (which is apparently so good that Kilgray will implement it in the next memoQ version, a rare exception to emulation that more often runs the other way)
• File format filters. (I have often used SDL filters to prepare content to translate in other environments)
• Project package sharing. (This is useful in any environment which offers it, but there is a need for all vendors to get off their butts and support interoperable package standards.)

Déjà Vu
See Victor Dewsbery's comment below. DVX used to be my favorite tool, one whose innovations have still to be matched in some respects by any of the competition. Its visionary software architect is arguably one the greatest contributors ever to the development of user friendly CAT tools.I am not personally familiar with the current version of the software at the present time nor with the server solution which was finally released.

memoQ
I spend far too much time on my blog talking about my fave features of this tool, so I asked a few others to have their say. One said that the things she found most helpful were

• PDF alignment (although it doesn't always work)
• The infinitely customizable interface - fonts, colors, sizes of windows, placement of windows, horizontal/vertical split, etc.
• Extensive [bilingual] export options (memoq bilingual, xliff, trados, two-column rtf)
• Lean software - version 5.0.62 is under 26 MB download, even smaller than version 3.0.37 which was nearly 35. Compare this to the bloat of SDL - currently 338 MB if my research is correct.
• "Duh" comments: very responsive support, frequent new releases and features, inexpensive

 *******

Do you use one of these tools or another and want to share the features which help you most to be productive or which just put a big smile on your face? Have your say in the comments.

.

Spamalot: desperate translation agencies

Soon after I started this blog in 2008 I found it necessary to take various measures to protect against spam comments. In the early days, the spam was a lot of the same that somehow makes its way to my e-mail inbox, usually real estate scams, offers of wondrous herbal mixtures to supersize various anatomical parts or God-knows-what in Chinese. Spam filters have become sophisticated enough to catch most of that now, so that the trash encountered most of the time now appears to have a bit more human intelligence behind it. That is, if you associate bottom-feeding translation agencies with human intelligence in some misguided way.

I have a definite impression that the competition in the race to the bottom for low prices and garbage quality has led some of these agencies to think they can improve their position in the global dog race by getting backlinks on popular translation blogs. This is done in comments on blog posts, sometimes as a link embedded in the rather vacuous text (as Rosetta once did to a New York agency rival's blog below) or in the link associated with the poster's name (as "Cassy" - surely not its real name - tried above on behalf of "LanguageTran"). What started out as the practice of a few sleazy Pakistani agencies years ago seems to have become routine for a number of UK and US bottom-feeders.

I've seen this happen on almost every translation blog I read. There are so many attempts made on my blog that I make it a point to check the name link on every comment that seems a bit "vacant", and a very large number get flushed to where they belong.

I'm not against comments from agency owners or personnel - quite the contrary. These people are a legitimate part of our business and have a lot to contribute to discussions - and many do. I love it when they or others have something interesting to say in a post discussion, particularly if they can point out an error on my part or add some useful point. But if the point of the visit is simply to post drivel in the hopes of getting a link to drive traffic to your agency's site or improve its search engine ranking, y'all can save yourselves the trouble because you will most likely not be getting through.