technocloud-public/nginx-unit
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

Python: Fix enabling of UTF-8 in some situations.

Browse Source 
There was a couple of reports of Python applications failing due to the
following type of error

File "/opt/netbox/netbox/netbox/configuration.py", line 25, in _import
     print(f"\U0001f9ec loaded config '{path}'")
UnicodeEncodeError: 'ascii' codec can't encode character '\U0001f9ec' in
position 0: ordinal not in range(128)

due to the use of Unicode text in the print() statement.

This only happened for python 3.8+ when using the "home" configuration
option as this meant we were going through the new PyConfig
configuration.

When using this new configuration method with the 'isolated' specific
API (for embedded Python) UTF-8 is disabled by default,
PyPreConfig->utf8_mode = 0.

To fix this we need to setup the Python pre config and enable utf-8
mode. However rather than enable utf-8 unconditionally we can set to it
to -1 so that it will use the LC_CTYPE environment variable to determine
whether to enable utf-8 mode or not. utf-8 mode will be enabled if
LC_CTYPE is either: C, POSIX or some specific UTF-8 locale. This is the
default utf8_mode setting when using the non-isolated PyPreConfig API.

Reported-by: Tobias Genannt <tobias.genannt@kappa-velorum.net>
Tested-by: Tobias Genannt <tobias.genannt@kappa-velorum.net>
Link: <https://peps.python.org/pep-0587/>
Link: <https://docs.python.org/3/c-api/init_config.html#c.PyPreConfig.utf8_mode>
Fixes: 491d0f70 ("Python: Added support for Python 3.11.")
Closes: <https://github.com/nginx/unit/issues/817>
Reviewed-by: Alejandro Colomar <alx@nginx.com>
Signed-off-by: Andrew Clayton <a.clayton@nginx.com>
This commit is contained in:
Andrew Clayton
2023-01-05 22:04:32 +00:00
parent a560cbf992
commit f3d05bba52
1 changed files with 14 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
src/python/nxt_python.c
View File

@@ -78,9 +78,23 @@ nxt_python3_init_config(nxt_int_t pep405)
PyConfig config; PyConfig config;
PyStatus status; PyStatus status;
nxt_int_t ret; nxt_int_t ret;
PyPreConfig preconfig;
ret = NXT_ERROR; ret = NXT_ERROR;
PyPreConfig_InitIsolatedConfig(&preconfig);
/*
* Determine whether to use UTF-8 mode or not, UTF-8
* will be enabled if LC_CTYPE is C, POSIX or some
* specific UTF-8 locale.
*/
preconfig.utf8_mode = -1;
status = Py_PreInitialize(&preconfig);
if (PyStatus_Exception(status)) {
return ret;
}
PyConfig_InitIsolatedConfig(&config); PyConfig_InitIsolatedConfig(&config);
if (pep405) { if (pep405) {