Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions examples-copier/.gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# Binaries
examples-copier
code-copier
*.exe
*.exe~
*.dll
Expand Down
24 changes: 15 additions & 9 deletions examples-copier/QUICK-REFERENCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,19 +108,25 @@ commit_strategy:
## Message Templates

### Available Variables
- `${rule_name}` - Copy rule name
- `${source_repo}` - Source repository
- `${target_repo}` - Target repository
- `${source_branch}` - Source branch
- `${target_branch}` - Target branch
- `${file_count}` - Number of files
- Custom variables from regex patterns
- `${rule_name}` - Copy rule name (e.g., "java-aggregation-examples")
- `${source_repo}` - Source repository (e.g., "mongodb/aggregation-tasks")
- `${target_repo}` - Target repository (e.g., "mongodb/vector-search")
- `${source_branch}` - Source branch (e.g., "main")
- `${target_branch}` - Target branch (e.g., "main")
- `${file_count}` - Number of files (e.g., "3")
- `${pr_number}` - Source PR number (e.g., "42")
- `${commit_sha}` - Source commit SHA (e.g., "abc123")
- Custom variables from regex patterns (e.g., `${lang}`, `${file}`)

### Examples
```yaml
commit_message: "Update ${category} examples from ${lang}"
pr_title: "Update ${category} examples"
pr_body: "Copying ${file_count} files from ${source_repo}"
pr_title: "Update ${lang} examples"
pr_body: |
Files updated: ${file_count} using ${rule_name} match pattern

Source: ${source_repo}
PR: #${pr_number}
```

## API Endpoints
Expand Down
4 changes: 1 addition & 3 deletions examples-copier/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -458,15 +458,13 @@ docker run -p 8080:8080 --env-file .env examples-copier

### Getting Started

- **[Configuration Guide](docs/CONFIGURATION-GUIDE.md)** - Complete configuration reference ⭐ NEW
- **[Configuration Guide](docs/CONFIGURATION-GUIDE.md)** - Complete configuration reference
- **[Pattern Matching Guide](docs/PATTERN-MATCHING-GUIDE.md)** - Pattern matching with examples
- **[Local Testing](docs/LOCAL-TESTING.md)** - Test locally before deploying
- **[Deployment Guide](docs/DEPLOYMENT.md)** - Deploy to production
- **[Deployment Checklist](docs/DEPLOYMENT-CHECKLIST.md)** - Step-by-step deployment checklist

### Reference

- **[Pattern Matching Cheat Sheet](docs/PATTERN-MATCHING-CHEATSHEET.md)** - Quick pattern syntax reference
- **[Architecture](docs/ARCHITECTURE.md)** - System design and components
- **[Troubleshooting](docs/TROUBLESHOOTING.md)** - Common issues and solutions
- **[FAQ](docs/FAQ.md)** - Frequently asked questions
Expand Down
2 changes: 1 addition & 1 deletion examples-copier/cmd/test-webhook/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -414,5 +414,5 @@ jobs:
- [Webhook Testing Guide](../../docs/WEBHOOK-TESTING.md) - Comprehensive testing guide
- [Local Testing](../../docs/LOCAL-TESTING.md) - Local development
- [Test Payloads](../../test-payloads/README.md) - Example payloads
- [Quick Reference](../../docs/QUICK-REFERENCE.md) - All commands
- [Quick Reference](../../QUICK-REFERENCE.md) - All commands

1 change: 0 additions & 1 deletion examples-copier/configs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,5 @@ examples-copier/

- [CONFIGURATION-GUIDE.md](../docs/CONFIGURATION-GUIDE.md) - Variable validation and reference
- [DEPLOYMENT.md](../docs/DEPLOYMENT.md) - Complete deployment guide
- [DEPLOYMENT-CHECKLIST.md](../docs/DEPLOYMENT-CHECKLIST.md) - Step-by-step checklist
- [LOCAL-TESTING.md](../docs/LOCAL-TESTING.md) - Local development guide

19 changes: 18 additions & 1 deletion examples-copier/configs/env.yaml.example
Original file line number Diff line number Diff line change
Expand Up @@ -111,10 +111,27 @@ env_variables:
# DEFAULT BEHAVIORS (OPTIONAL)
# =============================================================================
# System-wide defaults that individual config rules can override

DEFAULT_RECURSIVE_COPY: "true" # Default recursive copy behavior (default: true)
DEFAULT_PR_MERGE: "false" # Default auto-merge PRs without review (default: false)
DEFAULT_COMMIT_MESSAGE: "Automated PR with updated examples" # Default commit message (default: shown)

# =============================================================================
# GITHUB API CONFIGURATION (OPTIONAL)
# =============================================================================
# Fine-tune GitHub API retry and polling behavior

# GitHub API Retry Configuration
# Controls retry behavior when GitHub API calls fail due to eventual consistency
# GITHUB_API_MAX_RETRIES: "3" # Number of retry attempts (default: 3)
# GITHUB_API_INITIAL_RETRY_DELAY: "500" # Initial retry delay in milliseconds (default: 500)
# # Uses exponential backoff: 500ms, 1s, 2s, etc.

# PR Merge Polling Configuration
# Controls how long to wait for GitHub to compute PR mergeability
# PR_MERGE_POLL_MAX_ATTEMPTS: "20" # Max polling attempts (default: 20)
# PR_MERGE_POLL_INTERVAL: "500" # Polling interval in milliseconds (default: 500)
# # Total wait time = attempts × interval (default: ~10 seconds)

# =============================================================================
# TESTING / DEVELOPMENT OVERRIDES (DO NOT USE IN PRODUCTION)
Expand Down
53 changes: 45 additions & 8 deletions examples-copier/configs/environment.go
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,14 @@ type Config struct {
SlackUsername string
SlackIconEmoji string
SlackEnabled bool

// GitHub API retry configuration
GitHubAPIMaxRetries int
GitHubAPIInitialRetryDelay int // in milliseconds

// PR merge polling configuration
PRMergePollMaxAttempts int
PRMergePollInterval int // in milliseconds
}

const (
Expand Down Expand Up @@ -78,11 +86,15 @@ const (
AuditDatabase = "AUDIT_DATABASE"
AuditCollection = "AUDIT_COLLECTION"
MetricsEnabled = "METRICS_ENABLED"
SlackWebhookURL = "SLACK_WEBHOOK_URL"
SlackChannel = "SLACK_CHANNEL"
SlackUsername = "SLACK_USERNAME"
SlackIconEmoji = "SLACK_ICON_EMOJI"
SlackEnabled = "SLACK_ENABLED"
SlackWebhookURL = "SLACK_WEBHOOK_URL"
SlackChannel = "SLACK_CHANNEL"
SlackUsername = "SLACK_USERNAME"
SlackIconEmoji = "SLACK_ICON_EMOJI"
SlackEnabled = "SLACK_ENABLED"
GitHubAPIMaxRetries = "GITHUB_API_MAX_RETRIES"
GitHubAPIInitialRetryDelay = "GITHUB_API_INITIAL_RETRY_DELAY"
PRMergePollMaxAttempts = "PR_MERGE_POLL_MAX_ATTEMPTS"
PRMergePollInterval = "PR_MERGE_POLL_INTERVAL"
)

// NewConfig returns a new Config instance with default values
Expand All @@ -99,9 +111,13 @@ func NewConfig() *Config {
WebhookSecretName: "projects/1054147886816/secrets/webhook-secret/versions/latest", // default webhook secret name for GCP Secret Manager
CopierLogName: "copy-copier-log", // default log name for logging to GCP
GoogleCloudProjectId: "github-copy-code-examples", // default project ID for logging to GCP
DefaultRecursiveCopy: true, // system-wide default for recursive copying that individual config entries can override.
DefaultPRMerge: false, // system-wide default for PR merge without review that individual config entries can override.
DefaultCommitMessage: "Automated PR with updated examples", // default commit message used when per-config commit_message is absent.
DefaultRecursiveCopy: true, // system-wide default for recursive copying that individual config entries can override.
DefaultPRMerge: false, // system-wide default for PR merge without review that individual config entries can override.
DefaultCommitMessage: "Automated PR with updated examples", // default commit message used when per-config commit_message is absent.
GitHubAPIMaxRetries: 3, // default number of retry attempts for GitHub API calls
GitHubAPIInitialRetryDelay: 500, // default initial retry delay in milliseconds (exponential backoff)
PRMergePollMaxAttempts: 20, // default max attempts to poll PR for mergeability (~10 seconds with 500ms interval)
PRMergePollInterval: 500, // default polling interval in milliseconds
}
}

Expand Down Expand Up @@ -173,6 +189,14 @@ func LoadEnvironment(envFile string) (*Config, error) {
config.SlackIconEmoji = getEnvWithDefault(SlackIconEmoji, ":robot_face:")
config.SlackEnabled = getBoolEnvWithDefault(SlackEnabled, config.SlackWebhookURL != "")

// GitHub API retry configuration
config.GitHubAPIMaxRetries = getIntEnvWithDefault(GitHubAPIMaxRetries, config.GitHubAPIMaxRetries)
config.GitHubAPIInitialRetryDelay = getIntEnvWithDefault(GitHubAPIInitialRetryDelay, config.GitHubAPIInitialRetryDelay)

// PR merge polling configuration
config.PRMergePollMaxAttempts = getIntEnvWithDefault(PRMergePollMaxAttempts, config.PRMergePollMaxAttempts)
config.PRMergePollInterval = getIntEnvWithDefault(PRMergePollInterval, config.PRMergePollInterval)

// Export resolved values back into environment so downstream os.Getenv sees defaults
_ = os.Setenv(Port, config.Port)
_ = os.Setenv(RepoName, config.RepoName)
Expand Down Expand Up @@ -218,6 +242,19 @@ func getBoolEnvWithDefault(key string, defaultValue bool) bool {
return strings.ToLower(value) == "true"
}

// getIntEnvWithDefault returns the integer environment variable value or default if not set
func getIntEnvWithDefault(key string, defaultValue int) int {
value := os.Getenv(key)
if value == "" {
return defaultValue
}
var intValue int
if _, err := fmt.Sscanf(value, "%d", &intValue); err != nil {
return defaultValue
}
return intValue
}

// validateConfig checks if all required configuration values are set
func validateConfig(config *Config) error {
var missingVars []string
Expand Down
Loading