Issues with Mac OS

[pw-wasp]

Works fine on Fedora 35 and Debian 11 but no luck with MacOS Monterey. Always gives this error:

"IMAP server at localhost:1993 (unsecured) proxying imap.gmail.com:993 (SSL/TLS) encountered an SSL error - is the server's starttls setting correct? Current value: False"

I've tried everyting I can think of on client and server. Any suggentions?

Config:
[Server setup]

[IMAP-1993]
local_address = localhost
server_address = imap.gmail.com
server_port = 993

[SMTP-1465]
server_address = smtp.gmail.com
server_port = 465

[Account setup]

[--------@gmail.com]
permission_url = https://accounts.google.com/o/oauth2/auth
token_url = https://oauth2.googleapis.com/token
oauth2_scope = https://mail.google.com/
redirect_uri = http://localhost
client_id = --------------------------------apps.googleusercontent.com
client_secret = -----------------------------------

[simonrob]

The proxy definitely does work on macOS Monterey, so I'm not sure what is going on here. Do you have a firewall or other network monitoring/blocking tool running that could be interfering with the connection? Have you definitely set up your client to use an unsecured connection?

What output do you get if you try debugging via telnet (see the Troubleshooting section of the readme)

[pw-wasp]

Tried connecting with telnet, both imap and smtp, same error
Tried on MacOS 10.15.7 and 11.2.3 and 12.3 (latest), not virtual machines.
Could this be related to MacOS App Transport Security?

python3 emailproxy.py
2022-03-22 13:46:09: Starting IMAP server at localhost:1993 (unsecured) proxying imap.gmail.com:993 (SSL/TLS)
2022-03-22 13:46:09: Starting SMTP server at localhost:1465 (unsecured) proxying smtp.gmail.com:465 (SSL/TLS)
2022-03-22 13:46:09: Initialised Email OAuth 2.0 Proxy - listening for authentication requests
2022-03-22 13:46:27: IMAP server at localhost:1993 (unsecured) proxying imap.gmail.com:993 (SSL/TLS) encountered an SSL error - is the server's starttls setting correct? Current value: False
2022-03-22 13:47:02: SMTP server at localhost:1465 (unsecured) proxying smtp.gmail.com:465 (SSL/TLS) encountered an SSL error - is the server's starttls setting correct? Current value: False

telnet localhost 1993
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

• BYE IMAP server at localhost:1993 (unsecured) proxying imap.gmail.com:993 (SSL/TLS) encountered an SSL error - is the server's starttls setting correct? Current value: False
  Connection closed by foreign host.

telnet localhost 1465
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
221 SMTP server at localhost:1465 (unsecured) proxying smtp.gmail.com:465 (SSL/TLS) encountered an SSL error - is the server's starttls setting correct? Current value: False
Connection closed by foreign host.

[simonrob]

Given that this is happening before the connection is set up, it is very hard to debug remotely without being able to replicate it (which I can't). The proxy certainly does work on all of those macOS versions.

You mentioned that the proxy worked for you elsewhere - what happens if you copy the proxy's configuration file from those systems to one of the macOS ones that is presenting this issue?

Another thing that would help debug would be to get the full error details rather than the proxy's message. You can do this by making a minor edit to the script, inserting traceback.print_exc() after this line and adding import traceback with the other imports at the start. Run the proxy with this change, and try to connect using telnet again, then paste the error message here - hopefully it will reveal a little more about what is going on.

[pw-wasp]

Thanks for the suggestions.

python3 emailproxy.py
2022-03-23 07:37:19: Starting IMAP server at localhost:1993 (unsecured) proxying imap.gmail.com:993 (SSL/TLS)
2022-03-23 07:37:19: Starting SMTP server at localhost:1465 (unsecured) proxying smtp.gmail.com:465 (SSL/TLS)
2022-03-23 07:37:19: Initialised Email OAuth 2.0 Proxy - listening for authentication requests
2022-03-23 07:37:51: IMAP server at localhost:1993 (unsecured) proxying imap.gmail.com:993 (SSL/TLS) encountered an SSL error - is the server's starttls setting correct? Current value: False
Traceback (most recent call last):
File "/Users/dellz/Desktop/email-oauth2-proxy-main/emailproxy.py", line 895, in handle_accepted
new_server_connection = server_class(socket_map, self.server_address, address, self,
File "/Users/dellz/Desktop/email-oauth2-proxy-main/emailproxy.py", line 754, in init
super().init('IMAP', socket_map, server_address, connection_info, proxy_parent, custom_configuration)
File "/Users/dellz/Desktop/email-oauth2-proxy-main/emailproxy.py", line 687, in init
self.connect(self.server_address)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/asyncore.py", line 331, in connect
err = self.socket.connect_ex(address)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1347, in connect_ex
return self._real_connect(addr, True)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1333, in _real_connect
self.do_handshake()
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)

---

FROM THIS OUTPUT I WAS ABLE TO ZERO IN ON A FIX >>>> "Install Certificates.command"

/Applications/Python\ 3.9/Install\ Certificates.command ; exit;
-- pip install --upgrade certifi
Collecting certifi
Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
|████████████████████████████████| 149 kB 3.4 MB/s
Installing collected packages: certifi
Successfully installed certifi-2021.10.8
WARNING: You are using pip version 21.2.4; however, version 22.0.4 is available.
You should consider upgrading via the '/Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9 -m pip install --upgrade pip' command.
-- removing any existing file or link
-- creating symlink to certifi certificate bundle
-- setting permissions
-- update complete

Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.

[Process completed]

THE PROXY NOW WORKS!

[simonrob]

Excellent! Thanks for resolving this and reporting back.

Just to summarise for others encountering this problem on macOS: the issue here was that the Python installation (e.g., from python.org, homebrew, etc) did not have properly configured root certificates. This can be resolved by running Install Certificates.command, which can be found found in the Python folder in your Applications directory. The script is also available in the cpython repository.

[qianbinbin]

I was using python3 installed with brew. After I accidentally uninstalled certifi (also with brew) which I thought is not used any more, it asked me to remove /usr/local/etc/ca-certificates, and I did it.

So the error led me here... After reinstalling certifi with brew or pip again and again it just didn't work.

Then I checked the ruby scripts of brew, it turned out that cert under that dir are from ca-certificates, which is used by openssl@3, which is also required by python3.

What I need to do is reinstalling ca-certificates and openssl@3. The later one will create a symlink to the cert of ca-certificates.

The prompt of brew is quite misleading, hope no one will run into the question like me 😄