Just to confirm, this is the color picker tool you're referring to, not opening the color chooser dialog?
Currently you could use K to switch to the color picker tool, and if you have After select set to Switch to previous tool it'll always jump back to your current tool immediately after a click

Yes, the regular tool and not the dialog menu that opens.
Altho it would be nice to have the option to do this using ALT instead of using After select as a lot of media editing programs use that key for a color picker.

I also would like this. It's really convenient to just press alt, select a color, release and keep penciling.

For me in particular, I have a split keyboard, so 'K' is cumbersome to get to with my left hand while I mouse with my right.