replace string methods with unicode variants

[Elijahlorden 1]

The default string methods do not support unicode characters correctly (unicode.char(0x25A0):len() returns 3, for example).  Is it possible to replace the default string methods so that :len() references unicode.length()?  If not, is is possible to add methods for the unicode functions (perhaps :ulen(), :usub())?  I looked but could not find an answer to this.

[CptMercury 6]

Hey,

7 hours ago, Elijahlorden said:

But getmetatable does not return the string metatable.  I am assuming this is because the string metatable is protected for some reason.

you are correct, string metatables are protected, so you can‘t change or add functions.

I assume, unicode.len(string) is not what you want, or is it?

In case you want to have an object, you could interact with, you could build one.

local unicode = require"unicode"
local ucode = {}

local mt = {
  __len = function(self)
    return unicode.len(self.string)
  end
}

function ucode.new(str)
  local obj = {
    string = str
  }
  return setmetatable(obj, mt)
end

return ucode

Sorry for the code being just plane text, I’m on my mobile and dont know how to insert some code properly so i‘ll will make a proper one as soon as i get to my pc.

You then add all the missing functions of the unicode or string lib that you want to the metatable, and then you are good to go.

Btw, with the __len function, you should be able to call #objname, and it should return the correct length.

[Elijahlorden 1]

I have attempted the following:

local mt = rawget(getmetatable(''), "__index")
function mt:ulen() return unicode.len(self) end

But getmetatable does not return the string metatable.  I am assuming this is because the string metatable is protected for some reason.

[Elijahlorden 1]

I just wanted to add the unicode functions to the string object, not to create a string wrapper (Although that is a good idea, it seems like alot of overhead just for convenience methods).  I'll probably just add global references to the unicode functions.

[Molinko 43]

A combo of a wrapper and a custom program env might be able to do what you want but you're right that it might just be just a lot of overhead for a few convenience functions.. It also wouldn't cover cases on raw string values like ("yolo"):len()

[payonel 21]

it is better to think of these string functions as raw byte array methods, and our unicode methods as utf8 sequence methods

string.len is not a count of glyphs rendered on the screen, rather it is the number of chars in the string. a utf8 sequence may be one or more chars for a single glyph

string.sub selects a sequence of chars in a byte stream from [1, string.len]

unicode.len can be used for all utf8 strings to measure the number of logical utf8 sequences/glyphs

unicode.wlen measures the physical rendered width of a rendered utf8 string

unicode.sub deals with positions of sequence sets or glyphs. these indexes are in the range of [1, unicode.len]