Adapt Redpanda Migrator cookbook for Cloud #295
Conversation
|
Important Review skippedAuto incremental reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the 📝 WalkthroughWalkthroughThe PR updates the Redpanda Migrator cookbook documentation. It reorganizes content using env-cloud conditionals to separate Cloud and non-Cloud instructions, adjusts formatting (e.g., [source,bash] blocks, headings), and adds Cloud-specific guidance for endpoints, authentication, running pipelines (rpk connect run ...), and monitoring (OpenMetrics/Redpanda Cloud). It updates sample commands and provisioning steps to align with Cloud usage, refines offset remapping explanations, and removes non-essential references. Notes such as Brave browser snippet limitations were added. No code interfaces are changed; all edits are confined to documentation text and examples. Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Assessment against linked issues
Assessment against linked issues: Out-of-scope changes
Possibly related PRs
Suggested reviewers
✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
✅ Deploy Preview for redpanda-connect ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (3)
modules/cookbooks/pages/redpanda_migrator.adoc (3)
456-469: Cloud source (Kafka) missing TLS in migrator input; SASL config implies SASL_SSL.The earlier config.properties uses security.protocol=SASL_SSL, but this block configures SASL without TLS, which will fail against most managed Kafka endpoints.
redpanda_migrator: seed_brokers: [ "source.cloud.kafka.com:9092" ] @@ start_from_oldest: true + tls: + enabled: true sasl: - mechanism: SCRAM-SHA-256 username: kafka password: testpass
479-488: Cloud destination (Redpanda Cloud) missing TLS in migrator output.rpk examples use -X tls.enabled=true, but the migrator output lacks TLS. Add it to avoid connection failures.
redpanda_migrator_bundle: redpanda_migrator: seed_brokers: [ "destination.cloud.redpanda.com:9092" ] max_in_flight: 1 + tls: + enabled: true sasl: - mechanism: SCRAM-SHA-256 username: redpanda password: testpass
632-644: Cloud destination consumer missing TLS.As with other Cloud connections, enable TLS for kafka_franz to match SASL SCRAM usage.
input: kafka_franz: seed_brokers: [ "destination.cloud.redpanda.com:9092" ] @@ consumer_group: foobar + tls: + enabled: true sasl: - mechanism: SCRAM-SHA-256 username: redpanda password: testpass
🧹 Nitpick comments (6)
modules/cookbooks/pages/redpanda_migrator.adoc (6)
28-28: Upgrade this limitation to an admonition and double-check current support status.Make the limitation stand out and ensure it's still accurate for 2025-08-27.
-The Redpanda Migrator does not support migrations to or from Redpanda Cloud Serverless (multi-tenant) clusters. +IMPORTANT: Redpanda Migrator does not support migrations to or from Redpanda Cloud Serverless (multi-tenant) clusters.
104-108: Prefer Docker Compose v2 syntax.Most environments alias docker-compose, but v2 standardizes on “docker compose”. Minor polish.
-docker-compose -f docker-compose.yml up --force-recreate -V +docker compose -f docker-compose.yml up --force-recreate -V
118-133: Add a secrets handling note for sample credentials.These hardcoded creds are fine for a demo, but we should steer users toward secrets in Cloud.
---- broker: destination.cloud.redpanda.com:9092 schema registry: https://schema-registry-destination.cloud.redpanda.com:30081 username: redpanda password: testpass ---- + +NOTE: Example endpoints and credentials are placeholders for this tutorial. In Redpanda Cloud, store secrets in the pipeline configuration using the Secrets UI and avoid committing credentials to source control.
136-155: Clarify replication factor expectations for the sample Kafka cluster.Using --replication-factor=3 implies at least three brokers. Add a note to reduce confusion for single-node test clusters.
kafka-topics.sh --bootstrap-server source.cloud.kafka.com:9092 --command-config config.properties --create --if-not-exists --topic foo --replication-factor=3 --partitions=2 @@ kafka-topics.sh --bootstrap-server source.cloud.kafka.com:9092 --command-config config.properties --create --if-not-exists --topic bar --replication-factor=3 --partitions=2 + +# NOTE: Set --replication-factor to match your source cluster size (use 1 for single-broker test clusters).
620-624: Explicitly tell Cloud users to stop the source consumer pipeline before starting the destination one.The prose above mentions stopping the source consumer, but the Cloud UI steps don’t include it. Add a first step.
ifdef::env-cloud[] -. Go to the **Connect** page on your cluster and click **Create pipeline**. +. On the **Connect** page, stop the `read_data_source` pipeline you created earlier. +. Go to the **Connect** page on your cluster and click **Create pipeline**.
668-671: US spelling consistency: “minimises” → “minimizes”.Minor copy edit to align with US English used elsewhere.
-Although the timestamp-based approach doesn't guarantee exactly-once delivery, it minimises the likelihood of message duplication and avoids the need for complex and error-prone offset remapping logic. +Although the timestamp-based approach doesn't guarantee exactly-once delivery, it minimizes the likelihood of message duplication and avoids the need for complex and error-prone offset remapping logic.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
modules/cookbooks/pages/redpanda_migrator.adoc(20 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
- GitHub Check: Redirect rules - redpanda-connect
- GitHub Check: Header rules - redpanda-connect
- GitHub Check: Pages changed - redpanda-connect
🔇 Additional comments (1)
modules/cookbooks/pages/redpanda_migrator.adoc (1)
239-240: Validate compute unit sizing statement.“One compute unit is equivalent to 0.1 CPU and 400 MB of memory” may have changed. Please confirm with current Cloud docs.
Description
Resolves https://redpandadata.atlassian.net/browse/DOC-739
Page previews
Checks