• apoisel@discuss.tchncs.de
    link
    fedilink
    arrow-up
    127
    arrow-down
    2
    ·
    8 months ago

    These errors were much more common before Unicode encodings were in broad use. Unicode pretty much solved this.

    • wizardbeard@lemmy.dbzer0.com
      link
      fedilink
      English
      arrow-up
      19
      ·
      8 months ago

      Only if it’s enabled by default, or the dev knows to enable it.

      I had a lot of weird problems processing some info with names in Powershell until I found out that Powershell doesn’t default to unicode format when shoving output into files. You can easily specify the encoding, but if you don’t it replaces any non-ascii characters with “?” by default, so it’s not even immediately obvious that there’s an incorrect character, as it just silently substitutes a valid one.

      • voxel@sopuli.xyz
        link
        fedilink
        arrow-up
        2
        ·
        8 months ago

        it uses big-endian utf-16 with BOM by default unless you upgrade to PowerShell 7

    • fibojoly@sh.itjust.works
      link
      fedilink
      arrow-up
      5
      ·
      8 months ago

      I like your enthusiasm. I remember when I believed the same. The last 16 years have clearly shown this is not the case.

    • Norgur@fedia.io
      link
      fedilink
      arrow-up
      7
      arrow-down
      3
      ·
      8 months ago

      No it hasn’t. It has just pushed them out of sight for English natives.

      • apoisel@discuss.tchncs.de
        link
        fedilink
        arrow-up
        28
        ·
        8 months ago

        Can’t confirm that. In the 90s encodings were a nightmare. ISO-8859-1, ISO-8859-15, CP1252, IBM850, … If you tried to build a website with an upload form, you’d get the most bizarre encodings and there was no way to reliably distinguish them. I’m not an English native, my world is full of umlauts and s-z ligatures. Things got A LOT better in the last years, thanks to Unicode encodings.

    • PoolloverNathan@programming.dev
      link
      fedilink
      arrow-up
      2
      ·
      8 months ago

      Still needs to be widely used. It took me about an hour to figure out that my encoding issues were because of Vim being in latin1, another to figure out how to change that, and a third to realize that screen also wasn’t in UTF-8 mode.