-
Notifications
You must be signed in to change notification settings - Fork 6.6k
Add Managed Kafka Connect Connectors Examples #13522
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Add Managed Kafka Connect Connectors Examples #13522
Conversation
clusters * Adds code examples for creating, deleting, getting, listing and updating Managed Kafka Connect clusters
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This commit adds examples for creating Managed Kafka Connect connectors: * MirrorMaker connector * BigQuery sink connector * Cloud Storage sink connector * Pub/Sub sink connector * Pub/Sub source connector
Warning Gemini encountered an error creating the summary. You can try again by commenting |
Here is the summary of changes. You are about to add 5 region tags.
This comment is generated by snippet-bot.
|
managedkafka/snippets/connect/connectors/create_mirrormaker_connector.py
Outdated
Show resolved
Hide resolved
managedkafka/snippets/connect/connectors/create_mirrormaker_connector.py
Outdated
Show resolved
Hide resolved
managedkafka/snippets/connect/connectors/create_mirrormaker_connector.py
Outdated
Show resolved
Hide resolved
managedkafka/snippets/connect/connectors/create_mirrormaker_connector.py
Outdated
Show resolved
Hide resolved
As per the [Authoring Guide] (https://github.com/GoogleCloudPlatform/python-docs-samples/blob/main/AUTHORING_GUIDE.md#dependencies), each sample is required to have a requirements.txt file that lists the dependencies needed to run the sample.
Remove timeout to align with Managed Kafka Cluster update example: https://github.com/GoogleCloudPlatform/python-docs-samples/blob/cdae4cacfe8f9612e554af11ef72bc8d34765ada/managedkafka/snippets/clusters/update_cluster.py#L60
It looks like the linting here is failing because both the managedkafka/snippets and managedkafka/snippets/connect/connectors have requirements.txt files, which causes linting to run in both locations Google linting style requires that imports from local files are grouped separately. But when the linting is running from a parent folder, it doesn't see those local files, so the lint from the parent folder fails due to child folder local imports. It looks like #13515 also introduced a new requirements.txt file, so there's no longer just one requirements file for the whole snippets folder. Since this PR adds a requirements.txt file that duplicates the parent requirements.txt, I'm going to remove it from this PR. This will mean:
|
Note: This PR builds on top of the work in PR #13515 and should not be merged until that one is complete. The "Files changed" tab currently includes changes from the dependency.
This commit adds examples for creating Managed Kafka Connect connectors:
Description
Fixes https://b.corp.google.com/issues/430087669
Note: Before submitting a pull request, please open an issue for discussion if you are not associated with Google.
Checklist
nox -s py-3.9
(see Test Environment Setup)nox -s lint
(see Test Environment Setup)