目录
前言
在计算机中,我们如何表示数字,英文字母(A ~ Z / a ~ z),中文,符号(, .?<>),等各种符号呢?
- -
- 人们最终想到了对这些字符进行编码,比如在ASCII编码中,用一个字节的大小(8个bit位,故可表示的数量是 2^8 = 256个字符,但只用了 128 个,即(0 - 127))
- 比如:65 就代表 英文字母A,48 代表 数字0。
- 除此之外,还有很多 非打印字符(换行符、制表符等)以及 控制字符(退格、响铃等)控制编码。
- 本文中你将了解到:
- 常见的字符编码格式
- 字节序标记: 大端方式(Big endian) / 小端方式(Little endian)
- 带BOM是什么意思?
常见的字符编码格式
ASCII
- 单字节编码,使用一个字节的大小来表示一个字符(1Byte = 8 bit,故可表示的数量为 2^8 = 256个)到目前为止共定义了128个字符(0 - 127)。
- 单单我们中文常用的字符就不止256个,因此ASCII在全世界范围肯定是不够用的,因此后来才出来了很多扩展的编码格式。
- 如今的其他编码格式基本都兼容ASCII。即可理解为在它身上进行扩展。
ANSI
- American National Standards Institute (美国国家标准学会),由这个标准学会制订的一种编码规则。
- 使用 1 - 4 个字节来代表一个字符。ANSI在不同语言系统上代表不同的编码。不同 ANSI 编码之间互不兼容。
- 在 简体中文win系统 中代表 GBK
- 在 日文win系统 中代表 Shift_JIS
GBK / GB-2312 / GB18030
- 这几个就和我们中国密切相关了。
- GB2312:对 ANSI 的简体中文扩展。GB2312共收录了七千个字符。
- GBK:(即 国G标B 扩K展)由于GB2312支持的汉字太少而且不支持繁体中文,所以GBK对GB2312进行了扩展,以支持繁体中文和更多的字符,GBK共支持大概22000个字符。
- GB18030:在GBK的基础上又增加了藏文、蒙文、维吾尔文等主要的少数民族文字。
Unicode
- 由于(ANSI)各国的编码不同造成交流传输不便,ISO 打算废除所有的地区性编码方案,建立一个全球性的编码方案,并把所有字符都编码进去,称之为 "Universal Multiple-Octet Coded Character Set",简称 UCS(ISO10646)。
- 同时又有 unicode.org 这个组织也制定了自己的全球性编码 unicode,自从unicode2.0开始,unicode采用了与UCS相同的字库和字码。
- 现在用的是UCS-2,即2个字节编码,而UCS-4是为了防止将来2个字节不够用才开发的。
- Unicode只是一个符号集,它只规定了符号的二进制代码,却没有规定这个二进制代码应该如何存储(使用的字节数 / 定长存储 / 变长存储)。怎样存储、传输这些编码,则是由UTF来实现。
UTF
- 在可以使用ASCII表示的字符时使用Unicode并不高效,因为Unicode比ASCII占用大一倍的空间,而对ASCII来说高字节对他毫无用处。为了解决这个问题,就出现了一些中间格式的字符集,他们被称为通用转换格式,即UTF(Unicode Transformation Format)。这其中就有如UTF-8这种变长编码。而让最常出现的字符编码尽量的短,即可使得总的存储和传输代价小很多,这也是哈夫曼压缩编码的思想。
- UTF-8、UTF-16、UTF-32 都是 Unicode 的一种实现。目前使用最为广泛的就是UTF-8。这些看名字差别就是后面的 -8 / -16 / -32。意义就是处理单元的比特位数(即一次拿到 8 / 8 = 1 ; 16 / 8 = 2;32 / 8 = 4个字节进行处理)。
- UTF-8: 变长编码方式。也就是用这种编码出来不同字符的长度不一定相同。比如UTF-8表示ASCII中的字符需要1个字节,表示中文需要3个字节。
- UTF-16:变长编码方式。
- UTF-32:定长4字节编码。
字节序标记
处理单元为多个字节时(一次拿到多个字节进行处理),如果不分字节顺序的话,那么就会出现解读错误。比如我们一次要处理四个字节 12 34 56 78,这四个字节是表示 0x12 34 56 78 还是表示 0x78 56 34 12?不同的解释最终表示的值也不一样。因此就有了大端方式和小端方式之分。
大端方式(Big endian)
- 高位字节存在低位地址
- 低位字节存在高位地址
小端方式(Little endian)
- 高位字节存在高位地址
- 低位字节存在低位地址
- 显然 UTF-16 和 UTF-32 是需要区分大小端方式的,因为它们一次处理拿到的字节数不止一个。
- UTF-8 不需要区分大小端。
带BOM(签名)是什么意思?
- 在有些软件中(如VS2019),把带BOM翻译为带签名。
- 我们已经介绍了字节序的大端和小端方式,但怎么让计算机知道这段文本是大端还是小端方式呢?
- 这就需要在文件开头为BOM(Byte Order Mark)的字符来表明文件是 大端 还是 小端方式 来存取。
- 注意:BOM是Unicode标准建议的,因此 UTF-16 和 UTF-32 编码存储的文件需要带 BOM。而GB2312尽管是双字节,但是没有BOM这个概念的。因为他出生太早了,那时候还没有这个概念。GBK也是没有BOM的。
- UTF-8 不需要区分字节序,当然也就不需要 BOM。但可以用BOM来表明该文件的编码方式是UTF-8。但实际上非常不建议这样做,尽管 Unicode 标准允许在 UTF-8 中使用 BOM,但 UTF-8 带BOM,是Win基于兼容性考虑独创的格式,linux默认不兼容。
Al-Ittihad https://saudi.al-ittihad-ar.com is one of the most famous football clubs in Saudi Arabia. Founded in 1927, the Saudi football giant has come a long way to the pinnacle of success.
Al-Nasr https://saudi.al-nassr-ar.com is one of the most famous football teams in the Kingdom of Saudi Arabia.
Luis Suarez https://inter-miami.luis-suarez-ar.com the famous Uruguayan footballer, ended his brilliant career in European clubs and decided to try his hand at a new challenge – Major League Soccer.
Thank you, I have recently been searching for information about this topic for ages and yours is the best I have discovered till now. But, what about the bottom line? Are you sure about the source?
грузовой подъемник для склада https://podemniki-gruzovye.ru
https://amoxildelivery.pro/# amoxicillin 500mg over the counter
Bayern Munich’s https://bayern.jamal-musiala-ar.com young midfielder, Jamal Musiala, has become one of the brightest talents in European football.
buy cipro no rx: ciprofloxacin 500mg buy online – cipro pharmacy
Thibaut Courtois https://real-madrid.thibaut-courtois-ar.com was born on May 11, 1992 in Belgium.
https://ciprodelivery.pro/# ciprofloxacin over the counter
Arsenal https://arsenal.mesut-ozil-ar.com made a high-profile signing in 2013, signing star midfielder Mesut Ozil from Real Madrid.
Thank you for great content. Hello Administ. Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut
Good info. Lucky me I reach on your website by accident, I bookmarked it. Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut
Thank you great posting about essential oil. Hello Administ . Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut
Thank you for great content. Hello Administ. Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut
Thank you for great content. Hello Administ. Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut
EuroAvia24.com – Cheap flights, hotels and transfers around the world!
Rodrigo Goes https://real-madrid.rodrygo-ar.com better known as Rodrigo, is one of the brightest young talents in modern football.
The Saudi Football League https://saudi-arabian-championship.saudi-pro-league-ar.com known as the Saudi Professional League, is one of the most competitive and dynamic leagues in the world.
In an era when many young footballers struggle to find their place at elite clubs, Javi’s https://barcelona.gavi-ar.com story at Barcelona stands out as an exceptional one.
http://amoxildelivery.pro/# amoxicillin cost australia
In recent years, the leading positions in the Spanish https://spanish-championship.laliga-ar.com championship have been firmly occupied by two major giants – Barcelona and Real Madrid.
The Italian football championship https://italian-championship.serie-a-ar.com known as Serie A, has seen an impressive revival in recent years.
The road to the Premier League https://english-championship.premier-league-ar.com begins long before a team gets promoted to the English Premier League for the first time
The golf https://arabic.golfclub-ar.com industry in the Arab world is growing rapidly, attracting players from all over the world.
http://amoxildelivery.pro/# where to get amoxicillin over the counter
order amoxicillin 500mg: price of amoxicillin without insurance – amoxicillin 500mg buy online canada
Thank you for great information. Hello Administ . Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut
I really love to read such an excellent article. Helpful article. Hello Administ . Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut
Thank you for content. Area rugs and online home decor store. Hello Administ . Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut
After all, what a great site and informative posts, I will upload inbound link – bookmark this web site? Regards, Reader.Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut
http://paxloviddelivery.pro/# paxlovid buy
http://clomiddelivery.pro/# how to get generic clomid online
Manny Pacquiao https://filipino-boxer.manny-pacquiao-ar.com is one of the most prominent boxers in the history of the sport.
Muhammad Ali https://american-boxer.muhammad-ali-ar.com is perhaps one of the most famous and greatest athletes in the history of boxing.
Mike Tyson https://american-boxer.mike-tyson-ar.com one of the most famous and influential boxers in history, was born on June 30, 1966 in Brooklyn, New York.
enterprise car rental Montenegro rental cars Montenegro
I love what you guys tend to be up too. This sort of clever work and
coverage! Keep up the amazing works guys I’ve included you
guys to my blogroll.
https://doxycyclinedelivery.pro/# doxycycline compare prices
You ought to be a part of a contest for one of the finest
websites online. I’m going to recommend this website!
Michael Jordan https://chicago-bulls.michael-jordan-ar.com is one of the greatest basketball players of all time, whose career with the Chicago Bulls is legendary.
дизайн интерьера онлайн на русском дизайн интерьера дома
The Formula One World Championship https://world-circuit-racing-championship.formula-1-ar.com, known as the Formula Championship in motor racing, is the highest tier of professional motor racing.
After some difficult years in the late 2010s, Manchester United https://england.manchester-united-ar.com returned to greatness in English football by 2024.
https://ciprodelivery.pro/# ciprofloxacin generic
doxycycline uk cost: doxycycline otc uk – doxycycline brand
Jennifer Lopez https://lets-get-loud.jenniferlopez-ar.com was born in 1969 in the Bronx, New York, to parents who were Puerto Rican immigrants.
Good info. Lucky me I reach on your website by accident, I bookmarked it. Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut
Thank you for great information. Hello Administ . Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut
Thank you for great article. Hello Administ .Seo Paketi Skype: By_uMuT@KRaLBenim.Com -_- live:by_umut