Note: This is a follow-up to this question on StackOverflow.
I have to write a wrapper in Python to an external API, accessible through HTTP. The code is supposed to be publicly available on GitHub.
For this reason, I thought, it would be nice if a person cloning this repository wouldn’t see tons of warnings. I opened my own code in PyCharm just to see if that was the case. It wasn’t.
However, since I’m afraid what these warnings are pointing at is a design issue, please let me start by showing (in a simplified way) the design I came with:
Firstly: There are two methods to authenticate the HTTP connection. For this reason, I have a
ConnectionBase abstract class, that is implemented by two concrete classes, each using one of the two available authentication methods. So far so good.
But here problems start. There is an
ApiClient class that is supposed to contain wrappers of all those API routes. This
ApiClient class, of course, has to make use of an instance of
ConnectionBase. In addition, since there are so many API calls to provide wrappers for, I thought I’d break them into categories.
Finally, this is how (roughly) the definition of
ApiClient looks like:
class ApiClient: def __init__(self, connection): self._connection = connection # forwarding method async def _make_call(*args, **kwargs): return await self._connection._make_call(*args, **kwargs) class _Category: def __init__(self, client): self._client = client # another forwarding method async def _make_call(*args, **kwargs): return await self._client._make_call(*args, **kwargs) class _SomeCategory(_Category): async def some_api_call(some_arg, other_arg): return await self._make_call(blah blah) async def other_api_call(some_arg, other_arg): if some_arg.some_condition(): other_arg.whatever_logic_here() return await self._make_call(yadda yadda) class _OtherCategory(_Category): async def yet_another_api_call(some_arg, other_arg, yet_another_arg): #... return await self._make_call # etc etc @property def some_category(self): return ApiClient._SomeCategory(self) @property def other_category(self): return ApiClient._OtherCategory(self)
In this fashion, assuming that the user of my lib wants to make a call to this external API and that
client is an instance of
ApiClient, they would type:
I believe that my use of underscores is clear: I’m preceding with an underscore all names that are not meant to be called by the end user of my lib. I thought this was the most important distinction: far more important than the distinction between variables private to a class: because the latter is nothing but an aesthetic issue, while the former is a usability issue: after all, exposing a (hopefully) clean, well-defined and intuitive public API is the very purpose of writing libraries!
Yet, PyCharm frequently complains that I’m accessing protected members of classes from outside of these classes. Apparently, and according to the linked SO question, the Pythonic understanding of a name preceded by an underscore is: This member is internal to this class and NOT the way I was understanding it, that is: This member is internal to this package.
So, all such lines are producing warnings:
await self._connection._make_call await self._client._make_call
etc etc in other places of my code.
But then: Am I supposed to clearly distinguish between methods supposed to be called by the user of my lib and methods supposed to be called only from within my lib (or users who know the internal workings of my package and know what they’re doing)?
If yes, then how if not by an underscore, which apparently means a different thing?
Well I don’t know, because as it is clear from my questions on this site, my knowledge of design patterns is underwhelming… But a hypothesis of how this situation should be understood rises up in my mind… A pretty radical hypothesis…
Maybe the correct interpretation is that there should be no methods that are intended to be called from outside of a class but not from outside of the package and/or by the end user?
I mean, I’m constantly seeing talks about the need to break dependencies, about how bad it is to have code that is so closely intermingled and tightly coupled… Perhaps such methods are examples of this unwanted coupling? And in all cases where I’d like to put an underscore in the beginning of a method name but PyCharm complains, this simply means that the existence of such a method constitutes a coupling in my code that should not be there and therefore there should not be such a method at all?
And I was so proud of myself that I made use of dependency injection 😛
On a more serious note: I have strong doubts if the above, dire interpretation is correct. In Uni I had a basic course on OO design. The instructor said something along the lines of, IIRC:
Please do remember that inheritance should be used in case of an is-a relationship, not a has-a relationship. An example of an extremely bad inheritance is a car inheriting from a wheel or from a gas pedal. Such cases should be handled by composition instead.
So, since an API client is not a connection used by this API client and since a category of an API call is not a client, I thought that it would be wrong to make API client inherit from a connection or to make a category inherit from the client – even though doing this would make PyCharm stop complaining and issuing warnings. Instead I used composition. However, if composition is used, I can’t see how can I get rid of methods that are intended to be called from within my lib, but not from within the class they’re defined in and not by the end user.
I suppose there must be a basic design principle I’m ignoring out of ignorance.
Could you enlighten me please?