MySQL Truncated Content UTF-8 Error
You might have encountered the following error:
: Error (E_UNKNOWN) :: Encountered an unexpected error
: ER_TRUNCATED_WRONG_VALUE_FOR_FIELD: Incorrect string value: '\xC7\xB7\xC3\xB8\xD0\xAF...
' for column 'metadata' at row 1
at Query.Sequence._packetToError (~/dkit-device-manager-poc/sr
c/node_modules/sails-mysql/node_modules/mysql/lib/protocol/sequences/Sequence.js:48:14)
at Query.ErrorPacket (~/dkit-device-manager-poc/src/node_modul
es/sails-mysql/node_modules/mysql/lib/protocol/sequences/Query.js:83:18)
at Protocol._parsePacket (~/dkit-device-manager-poc/src/node_m
odules/sails-mysql/node_modules/mysql/lib/protocol/Protocol.js:280:23)
at Parser.write (~/dkit-device-manager-poc/src/node_modules/sa
ils-mysql/node_modules/mysql/lib/protocol/Parser.js:73:12)
at Protocol.write (~/dkit-device-manager-poc/src/node_modules/
sails-mysql/node_modules/mysql/lib/protocol/Protocol.js:39:16)
at Socket.<anonymous> (~/dkit-device-manager-poc/src/node_modu
les/sails-mysql/node_modules/mysql/lib/Connection.js:96:28)
at Socket.emit (events.js:182:13)
at Socket.EventEmitter.emit (domain.js:441:20)
at addChunk (_stream_readable.js:283:12)
at readableAddChunk (_stream_readable.js:264:11)
at Socket.Readable.push (_stream_readable.js:219:10)
at TCP.onStreamRead [as onread] (internal/stream_base_commons.js:94:17)
Error related to utf-8. MySQL's utf8 permits only the Unicode characters that can be represented with 3 bytes in UTF-8. Verify
SELECT
`tables`.`TABLE_NAME`,
`collations`.`character_set_name`
FROM
`information_schema`.`TABLES` AS `tables`,
`information_schema`.`COLLATION_CHARACTER_SET_APPLICABILITY` AS `collations`
WHERE
`tables`.`table_schema` = DATABASE()
AND `collations`.`collation_name` = `tables`.`table_collation`
;
device latin1
ALTER TABLE device MODIFY COLUMN metadata LONGTEXT
CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL;
If you need to migrate some content from the previous encoding to UTF8 you can use the following:
mysqldump --add-drop-table <database_to_correct> | replace CHARSET=latin1 CHARSET=utf8 | iconv -f latin1 -t utf8 | mysql <database_to_correct>
Docker setup
Need to configure default MySQL setup so that we use UTF8. If you are using the official docker MySQL image you can actually set these values from the docker run
command:
docker run --name mymysql \
-d mysql:5.7 \
--character-set-server=utf8mb4 \
--collation-server=utf8mb4_unicode_ci