This is a known issue in both Mapbox GL JS and MapLibre GL JS.
This plugin only supports Arabic-language ligatures and contextual forms by piggybacking on a part of Unicode that’s only intended for compatibility with legacy character encodings. Translating the proper Unicode characters to this compatibility shim requires a huge lookup table. This has to happen before reversing the character order, as required by other Arabic-script languages and Hebrew, which requires additional data to correctly wrap lines and such. Even when compressed, the data would bloat the overall GL JS package by more than Mapbox was initially willing to accept.
At the time, there was an assumption that most maps only displays labels in a single language that uses a left-to-right writing system. That assumption breaks in an OSM context, where we expect more multilingual capability. Even something like OSM Americana, which tailors the labels to your preferred language, still needs to be able to correctly display native Arabic or Persian names below those labels.
It might be possible to delegate the Arabic transformations to the browser using a technique I’m pursuing for complex text in general. It needs more investigation. But for the time being, installing the RTL plugin on the website would at least get Arabic and Hebrew to a usable state.