python / calibre-debug / venv

It feels like a catch-22, but if you use the calibre api it requires its own binary (calibre-debug) and yet to code in a virtual environment (venv), you also have to use a specific binary, in this case the python binary created in the venv. Is it possible to use the calibre database within python within a virtual environment? Am I missing something maybe?

Any help would be appreciated.