SNOW-862000: Support for External Authentication while running from EMR using spark-magic #1637
Description
What is the current behavior?
Currently while trying to use the snowflake.connector.connect function, and setting the authenticator to externalbrowser, the process works by starting a socket server, and giving an sso url that needs to be opened in an external browser to authenticate.
While this authentication happens, the process is waiting for input on the line
This method of waiting for user input doesn't seem to work when we are trying to authenticate from a jupyter-notebook that is connected to an EMR through spark-magic. In this example, I tried it from AWS sagemaker.
This doesn't work with spark-magic because spark-magic doesn't support waiting for user input. Hence this call ends up throwing the following error:
An error was encountered:
EOF when reading a line
Traceback (most recent call last):
File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/__init__.py", line 51, in Connect
return SnowflakeConnection(**kwargs)
File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/connection.py", line 319, in __init__
self.connect(**kwargs)
File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/connection.py", line 590, in connect
self.__open_connection()
File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/connection.py", line 860, in __open_connection
self.authenticate_with_retry(self.auth_class)
File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/connection.py", line 1127, in authenticate_with_retry
self._authenticate(auth_instance)
File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/connection.py", line 1145, in _authenticate
password=self._password,
File "/usr/local/lib64/python3.7/site-packages/snowflake/connector/auth/webbrowser.py", line 167, in prepare
url = input("Enter the URL the SSO URL redirected you to: ")
EOFError: EOF when reading a line
One workaround is to read the data locally using the %%local magic command from spark-magic, and sending it to the spark cluster. But that is inefficient since it is not scalable for large datasources.
What is the desired behavior?
One suggestion that I have to solve this issue is, instead of waiting for user input directly, can we have another function to feed the redirect url.
Hence we can get the SSO url in one function call. That function will return the sso url, while the socket server is kept running.
A second function can be called once we have the redirect url, which then posts it to the socket server, completing the authentication process.
This workaround should help add support for spark-magic.
This is one of the ideas that I can think of to circumvent the limitation posed by spark-magic. If there is a much cleaner way to get this done, suggestions are welcome.
How would this improve snowflake-connector-python?
This would add support for spark-magic, and allow users to connecting to Spark clusters remotely to use this library and read from snowflake directly from inside the cluster.
References and other background
No response