Description
The PyDict C API has a bad history. PyDict_GetItem() ignores all exception: error on hash(), error on "key == key2", KeyboardInterrupt, etc. PyDict_GetItemWithError() was added to fix this design. Moreover, Python 3.9 and older allowed to call PyDict_GetItem() with the GIL released.
PyDict_GetItem() returns a borrowed reference which is usually safe since the dictionary still contains a strong reference to the request value. But in general, borrowed references are error prone and can likely lead to complex race conditions causing crashes:
- "Functions must not return borrowed references" says https://devguide.python.org/developer-workflow/c-api/index.html
- Unclear lifetimes of borrowed references capi-workgroup/problems#5
While PyDict_GetItem() calls can be quite easily replaced with PyObject_GetItem() which has a better API (return a new stong reference), developers usually prefer to still use the specialized PyDict API for best performance: avoid the minor overhead of type dispatching, Py_TYPE(obj)->tp_as_mapping->mp_subscript.
I propose adding PyDict_GetItemRef() and PyDict_GetItemStringRef() functions to the limited C API (version 3.13): replacements for PyDict_GetItem(), PyDict_GetItemWithError() and PyDict_GetItemString().
API:
int PyDict_GetItemRef(PyObject *mp, PyObject *key, PyObject **pvalue);
int PyDict_GetItemStringRef(PyObject *mp, const char *key, PyObject **pvalue);
PyDict_GetItemWithError() has another API issue: when it returns NULL, it can mean two things. It returns NULL if the key is missing, but it also returns NULL on error. The caller has to check PyErr_Occurred() to distinguish the two cases (to write correct code). See capi-workgroup/problems#1 Proposed API avoids this by returning an int: return -1 on error, or return 0 otherwise (present or missing key). Checking PyErr_Occurred() is no longer needed.
By the way, the public C API has no PyDict_GetItemStringWithError() function: using PyDict_GetItemWithError() with a char* key is not convenient. The _PyDict_GetItemStringWithError() function exists but it's a private C API.