Skip to content

feat: add symbol removal API with accept conditions #68

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

feat: add symbol removal API with accept conditions #68

wants to merge 1 commit into from

Conversation

mufeedvh
Copy link
Member

Summary

Changes

  • Added VMRemoveTokens function in vst_maker.go that removes tokens based on:
    • Pattern (optional)
    • Value1 (optional)
    • Symbol type (optional)
    • Accept condition (required if not ACCEPT_ALL)
  • Added vm_remove_tokens C export in c-shared.go for Ruby integration
  • Added comprehensive unit tests in vst_maker_test.go

API Usage

// Remove tokens with specific pattern and accept condition
err := varnam.VMRemoveTokens("m", "ം", VARNAM_SYMBOL_ANUSVARA, VARNAM_TOKEN_ACCEPT_IF_ENDS_WITH)

// Remove all tokens with pattern regardless of position
err := varnam.VMRemoveTokens("ka", "", VARNAM_SYMBOL_CONSONANT, VARNAM_TOKEN_ACCEPT_ALL)

Test plan

  • Unit tests added for various removal scenarios
  • Validation tests for error cases
  • Integration testing with Ruby schemes

Fixes #23

@mufeedvh
feat: add symbol removal API with accept conditions
fc2ad31 
- Add VMRemoveTokens function to remove tokens based on pattern, value, type and accept conditions
- Add vm_remove_tokens C export for Ruby FFI integration
- Add comprehensive unit tests for the removal functionality
- Addresses varnamproject#23

This allows fine-grained control over symbol generation by enabling
removal of specific symbols based on their position (starts_with,
ends_with, in_between).
mufeedvh added a commit to mufeedvh/schemes that referenced this pull request Jun 21, 2025
@mufeedvh
feat: add symbol removal support in scheme compiler
dd2d265 
- Add FFI binding for vm_remove_tokens function
- Add _remove_token base method for internal use
- Add removal methods for all symbol types:
  - remove_vowels
  - remove_consonants
  - remove_consonant_vowel_combinations
  - remove_anusvara
  - remove_visarga
  - remove_virama
  - remove_symbols
  - remove_numbers
  - remove_others
- Add test examples and Malayalam scheme example
- Complements varnamproject/govarnam#68

This enables scheme authors to remove specific symbols based on
their position (starts_with, ends_with, in_between), providing
fine-grained control over transliteration rules.

Example usage:
remove_anusvara({:accept_if => :ends_with}, "m" => ["മ്"])
@mufeedvh mufeedvh mentioned this pull request Jun 21, 2025
Open
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Allow symbol removal from VST in VST Maker

1 participant

@mufeedvh