The memoQ Web Search memory leak fix! (updated again)

A big thank you to Italian veterinary surgeon and translating colleague Claudio Porcellana, who solved the mystery of the memory leak which has plagued users of memoQ's Web Search for years now. While Kilgray developers busily work on alternative engines for fixing future versions, Dr. Porcellana used his head - as impatient Southern Europeans are wont to do.

The problem it seems is with troublesome Java applets on sites like Linguee. So he simply turned them off. And plugged the leak.

Kilgray currently uses an Internet Explorer component for memoQ Web Search, so here's the fix:

1. Start Internet Explorer and open Internet options in the Settings:
   
   
    
2. Go to the Security tab and click the Custom level button:
   
   
    
3. Then find the Scripting section and disable the Java applets:
   

Leave Active scripting (= JavaScript, etc.) enabled or you will mess up the search for some sites like LEO.

After I made this change, I tested memoQ Web Search. Instead of the usual steady increase in memory consumption I used to observe due to the infamous leak, everything remained rock stable, and all my site searches that I typically use for legal and scientific translation worked just fine.

This fix ought to work with all versions of memoQ since the introduction of the web search feature (in memoQ 2013 R2 I think it was). So thank you, Dr. Porcellana, for making our working lives a little less crash-prone!

UPDATE: Further testing has revealed (as noted in some comments below) that there is more to the story. I was puzzled that some people continued to experience the memory leak unless "active scripting" was active, and at Varga's request I tested again on my system (I was sure up until then that his troubles might be tied to a Hungarian system, but it turns out that is in fact not the case). To9 my astonishment, the problem re-appeared after it had been eliminated before after disabling the Java applet scripting alone. I had to turn off "active scripting" too to achieve stability. And then suddenly the problem went away again.

Puzzling, right? And annoying of course. And then an idea occurred to me, and I dug up my Linguee user account password and logged in to Linguee under my user name. I contribute a lot of terms when I search in other browsers so I have a lot of credit, and this credit is applied as searches without ads.

It's the advertising. Some ads seem to involve Java applets. Other ads do buggy things with scripts that do not use applets. And some ads do neither of these two things and cause no trouble.

Maybe an ad blocker applied to Internet Explorer will fix the problem for memoQ Web search until the changeover to Chromium occurs in the next version. [No, it does not, alas.] In the meantime, I will achieve stability for today's big job by staying logged in to my Linguee account!

YET ANOTHER UPDATE: As advertisements and the like have been identified as the real source of trouble, one user suggested substituting the Windows hosts file. This approach has a number of advantages apparently; it presumably de-craps your Internet connection by blocking sites that send troublesome content, communicate with spyware, etc. A better hosts file with instructions for where to put it is found at: http://someonewhocares.org/hosts/

Substituted hosts file on my Windows 10 system; the old file was backed up by re-naming it.

General settings for memoQ TMs

memoQ TM settings are found in the Resource Console, the Options and a project's Settings.
This is a very useful "light resource" which is well worth nearly every user's time.

To define the TM settings to be used in new projects, select a settings configuration under Tools > Options... >  Default resources > TM settings (in the row of icons) by marking its checkbox.

To define the default TM settings to be used in the project you have opened, go to Project home > Settings > TM settings (in the row of icons) and mark the checkbox for the desired project default.

Different settings for individual TMs in a project (for example to set higher or lower match criteria) may be applied by going to Project home > Translation memories, selecting the TM of interest, clicking the Settings command at the right of the window and choosing the settings to apply instead of the project's standard TM settings.

The General settings tab is the same for all currently supported versions of memoQ. Role options are included on another tab in memoQ 2013 R2, and the Project Manager editions of memoQ offer additional possibilities for filtering and/or applying penalties to content on a Filters tab.

Match thresholds
The first value here (minimum) controls the fuzzy percentage below which a match will not be displayed in the translation results pane at the upper right of the working translation window.

The "good match" threshold is relevant to pretranslation (though this is unfortunately not made obvious in the dialog). The default value of 95% is really too high and would only apply to matches with small differences in tags or numbers; since any small difference in words is penalized significantly in memoQ (something I find very helpful, as I can understand more quickly what differences to look for compared to working in Trados). I usually set my "good matches" to 80%.

Not a "good match" according to the memoQ TM default setting

Penalties
In my work, an alignment penalty, which is a deduction from the match rate of a translation unit created by feeding an alignment to a translation memory, does not make a lot of sense. This is because

• I almost never send alignments to a TM. Why bother? LiveDocs may be slower in pretranslation, but it provides context matching just like a TM, and you can actually read what you find in a concordance search in its original document context. TMs suck because you do not get the full context for your matching segment and are thus at greater risk for missing information which may be important for a translation. This is especially the case with short match segments.
• if I happen to be aligning a dodgy translation and want to send it to a TM, I'll put it in a "quarantine TM" which already has its own penalty.
• on those rare occasions when I might feed an alignment to a TM, it's because the content is going to a user of another CAT tool, and if that person uses Trados or another tool that can read XLIFF files or other available bilingual formats, I'll send the data as that instad, so it can be reviewed and modified more easily before feeding to a TM. This also gives the other person a bilingual reference with document context.
• alignment for TMs is soooooo 1990s!

User penalties: If you have the misfortune to share a TM with someone whose work you do not trust completely and you want to avoid letting that person's 100% and context match segments slip past you unnoticed, apply a suitable penalty for the level of "risk" that person represents. If you want to be sure that user's content never gets used in a pretranslation and never appears in the translation results pane, apply a whopping big penalty like 80%. Those segments not be shown or inserted but will still be there in a concordance search if you want them.

TM penalties: Sometimes a client provides you with a TM you do not trust completely, or you may have a "quarantine TM" with content of dubious quality. Or I might have a TM with good content in British English but need to deliver a translation in American English. Applying penalties to such TMs will reduce the priority of their matches and prevent 100% matches with inappropriate language from slipping past without more careful inspection. As in the case of user penalties, you can also apply a very large penalty to ensure that matches will never be displayed in the translation results pane or used in a pretranslation but still have the TM content available for concordance searches.

Adjustments
It seems to be a good idea generally to enable the adjustment of fuzzy hits and inline tags. In many (but not all) cases, this will correct small differences in numbers, punctuation, cases and inline tags.

The only significant effect I was able to determine in adjusting the inline tag strictness in my tests was that more permissive settings might count a match with different tags as a full match. While this might meet the requirements of some clients hoping to impose discount schemes, from a quality assurance perspective, this does not seem like a good idea, and I believe it is better to have a strict setting here to draw attention to differences and reduce the chance that errors might be overlooked.

memoQ TM settings: beware the Kilgray defaults!

memoQ 2013 R2 introduced a very significant change in the management of translation memory data which most users are likely not aware of. However, because the default behavior for information storage in translation memories was changed, it is important to be aware of this difference and what to do before your data are unacceptably compromised.

The screenshot above shows several different translations stored in my TM for the sentence in the second segment. In previous versions of memoQ, only one translation would be stored with the way this translation memory was configured. However, in memoQ 2013 R2, the role of the person editing the translation becomes an important part of the "context", and as a result, multiple translations can be stored for different roles. Personally, I find this a rather useless feature, because if I want to know previous translations for a segment, I consult the row history using the context menu. But I understand how in some processes, it may be desirable to maintain a record of translations entered by the translator and the first and second reviewer.

I have no use for these older translations, especially as these may contain errors (as seen in the example of the third entry in the screenshot). If I am proofreading my translation in a "reviewer" role and make changes, I want to overwrite the original entry in my TM and avoid the chance that its errors will be propagated in later work.

To avoid the problems that can result from this redundancy and preservation of errors in the translation memory, as of build 6.8.6 it is necessary for users to explicitly opt out of the current Kilgray TM settings defaults and create their own custom settings.

TM settings are "light resources" which can be managed in four places:

• The Resource Console,where settings can be created, edited, imported, exported, etc.
• The Options (Tools > Options... > Default resources > TM settings), where the default for new projects can also be set
• Project Settings (Project home > Settings > TM settings) in a specific project, where the default settings for the current project can be set
• Project home > Translation memories > (TM) > Settings where alternative TM settings can be specified for a particular translation memory selected in  project. This would be the case there you want to apply a special set of penalties to the content of that TM, for example.

The last tab of the default TM settings dialog looks like this:

To avoid the trouble of multiple, role-based entries being written to a TM, settings must be created in which the option to Store modifying user's role in the TM entries in not selected, and these custom settings must be applied to the primary translation memory in the project (by default or explicit selection).

Here's the "fast path" for staying out of trouble:

1. Go to Tools > Options > Default resources > TM settings and if you do not already have custom TM settings to edit, select and clone the default settings. Give them a suitable name like "My Own TM Settings".
2. Click the Roles tab and unmark the setting to store the user's role in TM entries.
3. Click OK.
4. Ensure that the checkbox next to these custom settings is marked so they will be applied to all new projects. Then click OK to exit the options.
5. In any currently open project to which the desired settings have not been applied, go to Project home > Settings > TM settings and select the desired settings as the default by marking the corresponding checkbox.

Multiple entries written to the TM when the roles are included will not be eliminated after the TM settings are corrected. They must be explicitly removed by editing the translation memory.

I hope that in the future Kilgray will reconsider these troublesome new default settings and make the new possibilities "opt-in" values in custom TM settings. But for now, users must actively change their settings and defaults if they want to avoid role-based additional TM entries. (The current version of the memoQ Help describes roles as being disabled here by default. Would that this were so!)

You can, of course, make other useful adjustments to your custom TM settings, such as defining what a "good" match is (for pre-translation) or adjusting the tag matching behavior or applying various kinds of penalties to reduce match values for content which might have quality problems. The memoQ Help offers guidance on these options.

Postscript:
Even after the settings are "fixed", "existing damage" in a TM caused by the storage of unwanted, role-based information is not repaired. Any messes will have to be cleaned up in the rather inadequate TM editor in memoQ or in an external TMX maintenance tool. At the present time, there is no "easy option" to clean up a large number of erroneous or redundant translations stored because of this role setting. This case unfortunately underscores the woefully inadequate maintenance facilities for translation memory resources in the current version of memoQ. Perhaps some of the sophisticated options developed for Kilgray's TM Repository will finally trickle down in some way in an integrated option with Language Terminal or some sophisticated filtering and editing options will be added directly to the desktop product so that users can finally maintain their TM data in a reasonable way. memoQ is, overall, the best option available to us for project work in most cases, and I recommend it to colleagues because I know they will be able to do most ordinary tasks with a minimum of grief and calls for help (or expressions of anger) directed to me. But in 2013 it is ridiculous that my ability to manage my TM in my tool of choice is inferior to what I could do when I started using Déjà Vu as my CAT tool 13 years ago. Please join me in encouraging Kilgray to raise their game - soon - with respect to translation memory maintenance by writing to support@kilgray.com and expressing your need for better data management! (And more sensible default TM settings, of course.)

Update:
It looks like this default problem may end with the 6.8.6 build. One of the key people involved with memoQ and its features has stated that "After the [next] update, the default TM settings resource will have 'Store modifying user’s name in TM' unticked." Excellent.

For cases where there may already be data problems from older, erroneous entries being retained, the following workaround was suggested:

• Export to TMX
• Start up 6.5 and import into an empty TM

In the process, memoQ 2013 (version 6.5) will ignore the role information in the TMX, and entries with the same source will not create duplicates; translations with a later timestamp will be preserved in the TM if there are duplicates in the TMX.

This still doesn't change the fact that we need better means of maintaining our data in memoQ, but it is good that once again, Kilgray has responded quickly to important concerns of its users and is on the way to solving the problem.