Discourse Translator - Pricing and Language Coverage

ezekielf · 2022-04-12T14:31:49.372Z

This seems like a good guideline for this multilingual forum.

Always feel free to post in your native language
When replying to a thread in a language differing from the original post, it is polite to also include a translation (machine or otherwise) of your text into the OP language.

Hungerburg · 2022-04-12T23:31:45.802Z

Maybe it was late and I was tired, as its now; Writing can be exhausting, to get everything right, writing in a foreign language is not the same as translating, it is thinking in a foreign language, and that does not make the task any easier.

Maybe this incident can be seen as a preview of what a truly multilingual forum will face. From toying around with what machine translation offers, I guess, I will have to adapt my writing a bit, to get a translation, that meets my expectations. This for English, no idea about languages that I have no command of in the least.

Implementation challenges: When I post in native language, will the post appear as English to me/others? Will it appear in whatever language my or their user agent (aka web-browser) claims? Will the source language text be shown in a tooltip on mouseover?

Information challenges: As a user, I think I’d not have time, to follow a category, where people from all over the world post topics, that I get to read translated. Machine translation surely makes life easier for moderators, but does it the same for users, that want to provide “Help and support” e.g?

On the controversial post above: Maybe language detection brings us a big part of what translation can do. If the OP (topic opener) sets the language of the topic, a tag can do that, preferably in the language and script gathered. So people can subscribe to receive posts to some category in some language.

(In case, this garners little attention, they can always try and post in a lingua franca, that might get them a wider audience.)

wolfgang8 · 2022-04-13T07:35:33.173Z

Yes, I agree, communication in foreign languages is not easy and often associated with misunderstandings. But these can usually be cleared up easily if there is a benevolent effort on all sides.

The upcoming internationalisation of communication is undoubtedly a challenge for the community. For me, it is an ongoing process in which solutions can be tried out and also discarded if it turns out that they don’t work. And with Discourse there is a powerful tool available that should enable us to realize this flexible approach.

Firefishy · 2022-04-13T23:17:33.348Z

I have installed the discourse translation extension. I have enabled Azure (Microsoft) as the translation provider.

Please update this thread with any feedback.

PS: The little “world” icon at the bottom of a non-UI language post is the translate button.

westnordost · 2022-04-13T23:41:42.230Z

Will it be monitored how much it is used after a few weeks so that the price doesn’t explode without noticing (and we have an estimate how much it will cost in the future)?

Firefishy · 2022-04-13T23:42:45.033Z

Yes, I will check in a few weeks the cost. I have $200 free available for 1 year in Azure.

yvecai · 2022-04-14T03:31:16.211Z

My first impression is really good.
Even if my first trial dutch->French is as suboptimal as I would have expected with a machine translation, I had my little ‘wow’ effect. Good job!
One have to be logged in, any way to default to English for non logged in user?

wolfgang8 · 2022-04-14T07:09:13.746Z

Great job, thank’s very much!

Sometimes I noticed translation frequency limitations and I had to wait a few seconds, but for free it’s ok.

Adam_Franco · 2022-04-14T12:08:17.575Z

Since the translation needs to be done on each post I very quickly started getting rate limited after about 4 clicks. :-/ Not a problem for testing the translations, but not a great experience if this would be the result long term.

Screen Shot 2022-04-14 at 8.00.18 AM1270×823 73.7 KB

TomH · 2022-04-14T12:19:30.839Z

Adam_Franco:

Since the translation needs to be done on each post I very quickly started getting rate limited after about 4 clicks

The rate limit is currently set to three translations per minute so I guess it depends how long the things you translate are and how long it takes to read them…

Adam_Franco · 2022-04-14T12:42:50.540Z

I wouldn’t imagine that such a limit would be a problem for the odd post in a mixed-language thread, but I was trying to read a thread that was all in dutch (which I can’t read) and so had to click to translate each little 3-sentence post in the thread.

If there was a way to translate all posts in a thread that would cut down on the number of clicks, but maybe that would have other issues in terms of translating from many source languages to one destination in a single larger request.

ezekielf · 2022-04-14T14:00:36.172Z

I was hoping that once a post had been translated Discourse would cache the translation for some time to minimize redundant API requests. As far as I can tell every time the translate button is clicked is a new request though. Can any of the sysadmins confirm or deny this?

westnordost · 2022-04-14T14:42:29.914Z

Zeke Farwell:

As far as I can tell every time the translate button is clicked is a new request though

(trying out auto-translate)

Wie kommst Du darauf? Klar ist die Übersetzung ein eigener Aufruf von Deinem Browser aus gesehen, aber zu Discourse.

Edit: Diesen Absatz habe ich später hinzugefügt nachdem ich den Text einmal habe automatisch übersetzen lassen. Mal sehen ob es entsprechend neu übersetzt wird.

ezekielf · 2022-04-14T15:39:21.802Z

westnordost:

Wie kommst Du darauf? Klar ist die Übersetzung ein eigener Aufruf von Deinem Browser aus gesehen, aber zu Discourse.

I clicked the translate button quickly on three different posts in order to trigger the rate limit message. Then in a separate tab clicked the translate button on the same three posts. I figure if the translations are getting cached then the rate limit message wouldn’t show up since a live API call to Bing translate would not be needed. The rate limit message does show up on the same posts so I’m assuming they aren’t getting cached. However, it’s easily possibly that things are working differently behind the scenes than how I’m imagining and the rate limit is simply tied to the button click regardless of any caching that may or may not be happening.

westnordost · 2022-04-14T17:07:22.522Z

Hm, clever. So either the caching doesn’t work, or the rate limiting is checked before it is checked whether a post is already cached.

westnordost · 2022-04-14T17:25:41.974Z

Okay, it looks like the rate limiting is checked first, here:

github.com

discourse/discourse-translator/blob/c6c80cd68b05cb93ee8bee80102dfd7fa0cfb687/plugin.rb#L37-L39

      
        
            if !current_user.staff?
              RateLimiter.new(current_user, "translate_post", SiteSetting.max_translations_per_minute, 1.minute).performed!
            end

And the check whether the translated text has already been saved to the database is done after that, here:

github.com

discourse/discourse-translator/blob/4344d75fbccc014a271d1be4cdb9396986bcf84e/services/discourse_translator/base.rb#L34-L44

      
        
            post_translated_custom_field = post.custom_fields[DiscourseTranslator::TRANSLATED_CUSTOM_FIELD] || {}
            translated_text = post_translated_custom_field[I18n.locale]
            
            
if translated_text.nil?
              translated_text = yield
            
            
  post.custom_fields[DiscourseTranslator::TRANSLATED_CUSTOM_FIELD] =
                post_translated_custom_field.merge(I18n.locale => translated_text)
            
            
  post.save!
            end

I guess a good solution would be to move the “checking whether something is in cache” logic out of the DiscourseTranslator::Base.from_custom_fields method and into DiscourseTranslator::TranslatorController.translate so the order of rate-limiting and cache-checking can be switched.

However, I won’t create a PR because I don’t know Ruby at all. Maybe someone with at least some basic knowledge in Ruby could get this done.

Map_HeRo · 2022-04-14T18:57:00.426Z

I have tried the translator on several posts and believe this is a good step forward in removing language barriers. Not yet a fully automated system but easy to handle and delivering reasonable translations. A bit stiff here and there but you can always get the point, at least for those languages I tried. As long as everyone contributing to a thread puts a little effort into correct wording this should do quite well.

For me this is a must in a forum which claims internationality … at the end of the day it’s a question of the price of course …

Map_HeRo · 2022-04-14T19:30:21.850Z

I have noticed the globe icon under my own post … finally I am able to translate my own message into my native language … this is the first time I really understand what I have been posting … I love it …

Firefishy · 2022-04-14T21:24:09.342Z

Adam_Franco:

Since the translation needs to be done on each post I very quickly started getting rate limited after about 4 clicks. :-/ Not a problem for testing the translations, but not a great experience if this would be the result long term.

Screen Shot 2022-04-14 at 8.00.18 AM1270×823 73.7 KB

The limit is 3 translations a minute. There is a tiny cost associated with each translation (to use the Azure Translate API). If 3 per minute is too low for general usage then I’m ok with increasing it a little.

Hungerburg · 2022-04-14T22:25:22.008Z

I clicked the World icon below my English post. At first, result seems fine, except… when there are words, that have several meanings. I have seen this before, the translation engine then sometimes selects the wrong branch, making the output mostly indecipherable.

But yes, this, the extra click and expanding space, looks a practical way to deal with stray foreign language posts in threads/topics.