forked from TanStack/db
-
Notifications
You must be signed in to change notification settings - Fork 0
[POC] PowerSync Integration #1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
stevensJourney
wants to merge
60
commits into
main
Choose a base branch
from
powersync
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
60 commits
Select commit
Hold shift + click to select a range
27a3728
wip: PowerSync collections
stevensJourney e88623c
Add support for transactions with multiple collection types
stevensJourney 352829e
Optimize transaction waiting
stevensJourney 1c75d3d
Improve test stability
stevensJourney 50f3383
Merge remote-tracking branch 'upstream/main' into powersync
stevensJourney a892acc
Improve cleanup behaviour
stevensJourney 7d9ff73
Add rollback test
stevensJourney d5b3d99
update dependencies
stevensJourney cc42e94
Add live query test
stevensJourney d1de549
Add docs for PowerSync collection
stevensJourney c0a212a
Merge branch 'main' into powersync
stevensJourney ccba6ef
Add Changeset
stevensJourney c887d90
Added schema conversion and validation
stevensJourney 860fa26
ensure observers are ready before proceeding with mutations
stevensJourney ffa68d1
Add logging
stevensJourney 79abf05
Implement batching during initial sync
stevensJourney 237ed35
Update log messages. Avoid requirement for NPM install scripts.
stevensJourney 8d489e9
Schemas Step 1: Infer types from PowerSync schema table.
stevensJourney 4692c8b
Support input schema validations with Zod
stevensJourney fb45f02
update readme
stevensJourney 7030117
Update doc comments. Code cleanup.
stevensJourney 829ce64
More doc cleanup
stevensJourney dd0cbc8
README cleanup
stevensJourney dc0b361
Merge branch 'main' into powersync
stevensJourney e26bf27
Cleanup tests
stevensJourney e207268
Update PowerSync dependencies
stevensJourney e94cadf
Properly constrain types
stevensJourney b7fc0ff
Allow custom input schema types
stevensJourney 8187c6d
Support `orderBy` and `limit` in `currentStateAsChanges` (#701)
kevin-dp 96ad9d3
Fix bug when moving an orderBy window that has an infinite limit (#705)
kevin-dp 36d2439
ci: Version Packages (#702)
github-actions[bot] af6a4e4
docs: document findOne method in live queries guide (#699)
KyleAMathews 5950583
Manual writes should validate against the synced store, not the combiβ¦
KyleAMathews 16dbfe3
fix(query-db-collection): respect QueryClient defaultOptions when notβ¦
KyleAMathews 5ab979c
ci: Version Packages (#711)
github-actions[bot] 3c9526c
fix: dedupe filtering for non-optimistic mutations (#715)
mpotter d8ef559
ci: Version Packages (#716)
github-actions[bot] 970616b
fix(collection): fire status:change event before cleaning up event haβ¦
KyleAMathews ac42951
ci: Version Packages (#718)
github-actions[bot] 518ecda
chore(deps): update all non-major dependencies (#724)
renovate[bot] c2a5c28
feat: add exact refetch targeting and improve utils.refetch() behavioβ¦
lucasweng 2d4d5e1
ci: Version Packages (#726)
github-actions[bot] fbfa75a
Support better schema type conversions
stevensJourney da9ec60
docuement deserialization errors
stevensJourney c439899
Fix typo in READMe
stevensJourney db3eae5
Add type to README example
stevensJourney 6738247
Feat: Add support for custom parsers/serializers in LocalStorage collβ¦
sadkebab 7b9c681
ci: Version Packages (#731)
github-actions[bot] 7e9a1d8
Fix flaky test (#735)
KyleAMathews 9e4cbef
Document how to destructure in Svelte (#733)
KyleAMathews f8a979b
Fix: Optimizer Missing Final Step - Combine Remaining WHERE Clauses (β¦
KyleAMathews 979a66f
Enable auto-indexing for nested field paths (#728)
KyleAMathews d2b569c
Investigate Size Change action minification (#736)
KyleAMathews cb25623
feat: Add paced mutations with timing strategies (#704)
KyleAMathews 48b8e8f
ci: Version Packages (#739)
github-actions[bot] fe165e5
update PowerSync packages
stevensJourney 15e981f
Merge remote-tracking branch 'upstream/main' into powersync
stevensJourney e4024a0
set author to POWERSYNC
stevensJourney 81230f9
rename serlization.ts β serialization.ts
stevensJourney 718c4f9
use MIT license
stevensJourney File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| --- | ||
| "@tanstack/powersync-db-collection": minor | ||
| --- | ||
|
|
||
| Initial Release |
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,365 @@ | ||
| # Implementation Plan for `useSerializedTransaction` with TanStack Pacer | ||
|
|
||
| Based on [GitHub issue #35](https://github.com/TanStack/db/issues/35), using @tanstack/pacer for strategy implementation across all 5 framework integrations. | ||
|
|
||
| ## Overview | ||
|
|
||
| Create a framework-agnostic core in `@tanstack/db` that manages optimistic transactions with pluggable queuing strategies powered by TanStack Pacer. Each framework package wraps the core with framework-specific reactive primitives. | ||
|
|
||
| ## Architecture Pattern | ||
|
|
||
| The core transaction logic stays in one place (`@tanstack/db`) while each framework provides its own wrapper using framework-specific reactive primitives. | ||
|
|
||
| ```typescript | ||
| // Core in @tanstack/db (framework-agnostic) | ||
| createSerializedTransaction(config) // Returns { mutate, cleanup } | ||
|
|
||
| // React wrapper | ||
| useSerializedTransaction(config) // Uses React hooks, returns mutate function | ||
|
|
||
| // Solid wrapper | ||
| useSerializedTransaction(config) // Uses Solid signals, matches useLiveQuery pattern | ||
|
|
||
| // Svelte/Vue wrappers | ||
| useSerializedTransaction(config) // Framework-specific implementations | ||
|
|
||
| // Angular wrapper | ||
| injectSerializedTransaction(config) // Uses Angular DI, follows injectLiveQuery pattern | ||
| ``` | ||
|
|
||
| ## Available Strategies (Based on Pacer Utilities) | ||
|
|
||
| ### 1. **debounceStrategy({ wait, leading?, trailing? })** | ||
|
|
||
| - Uses Pacer's `Debouncer` class | ||
| - Waits for pause in activity before committing | ||
| - **Best for:** Search inputs, auto-save fields | ||
|
|
||
| ### 2. **queueStrategy({ wait?, maxSize?, addItemsTo?, getItemsFrom? })** | ||
|
|
||
| - Uses Pacer's `Queuer` class | ||
| - Processes all transactions in order (FIFO/LIFO) | ||
| - FIFO: `{ addItemsTo: 'back', getItemsFrom: 'front' }` | ||
| - LIFO: `{ addItemsTo: 'back', getItemsFrom: 'back' }` | ||
| - **Best for:** Sequential operations that must all complete | ||
|
|
||
| ### 3. **throttleStrategy({ wait, leading?, trailing? })** | ||
|
|
||
| - Uses Pacer's `Throttler` class | ||
| - Evenly spaces transaction executions over time | ||
| - **Best for:** Sliders, scroll handlers, progress bars | ||
|
|
||
| ### 4. **batchStrategy({ maxSize?, wait?, getShouldExecute? })** | ||
|
|
||
| - Uses Pacer's `Batcher` class | ||
| - Groups multiple mutations into batches | ||
| - Triggers on size or time threshold | ||
| - **Best for:** Bulk operations, reducing network calls | ||
|
|
||
| ## File Structure | ||
|
|
||
| ``` | ||
| packages/db/src/ | ||
| βββ serialized-transaction.ts # Core framework-agnostic logic | ||
| βββ strategies/ | ||
| βββ index.ts # Export all strategies | ||
| βββ debounceStrategy.ts # Wraps Pacer Debouncer | ||
| βββ queueStrategy.ts # Wraps Pacer Queuer | ||
| βββ throttleStrategy.ts # Wraps Pacer Throttler | ||
| βββ batchStrategy.ts # Wraps Pacer Batcher | ||
| βββ types.ts # Strategy type definitions | ||
|
|
||
| packages/db/package.json # Add @tanstack/pacer dependency | ||
|
|
||
| packages/react-db/src/ | ||
| βββ useSerializedTransaction.ts # React hook wrapper | ||
|
|
||
| packages/solid-db/src/ | ||
| βββ useSerializedTransaction.ts # Solid wrapper (matches useLiveQuery pattern) | ||
|
|
||
| packages/svelte-db/src/ | ||
| βββ useSerializedTransaction.svelte.ts # Svelte wrapper | ||
|
|
||
| packages/vue-db/src/ | ||
| βββ useSerializedTransaction.ts # Vue wrapper | ||
|
|
||
| packages/angular-db/src/ | ||
| βββ injectSerializedTransaction.ts # Angular wrapper (DI pattern) | ||
|
|
||
| packages/*/tests/ | ||
| βββ serialized-transaction.test.ts # Tests per package | ||
| ``` | ||
|
|
||
| ## Core API Design | ||
|
|
||
| ```typescript | ||
| // Framework-agnostic core (packages/db) | ||
| import { debounceStrategy } from '@tanstack/db' | ||
|
|
||
| const { mutate, cleanup } = createSerializedTransaction({ | ||
| mutationFn: async ({ transaction }) => { | ||
| await api.save(transaction.mutations) | ||
| }, | ||
| strategy: debounceStrategy({ wait: 500 }), | ||
| metadata?: Record<string, unknown>, | ||
| }) | ||
|
|
||
| // mutate() executes mutations according to strategy and returns Transaction | ||
| const transaction = mutate(() => { | ||
| collection.update(id, draft => { draft.value = newValue }) | ||
| }) | ||
|
|
||
| // Await persistence and handle errors | ||
| try { | ||
| await transaction.isPersisted.promise | ||
| console.log('Transaction committed successfully') | ||
| } catch (error) { | ||
| console.error('Transaction failed:', error) | ||
| } | ||
|
|
||
| // cleanup() when done (frameworks handle this automatically) | ||
| cleanup() | ||
| ``` | ||
|
|
||
| ## React Hook Wrapper | ||
|
|
||
| ```typescript | ||
| // packages/react-db | ||
| import { debounceStrategy } from "@tanstack/react-db" | ||
|
|
||
| const mutate = useSerializedTransaction({ | ||
| mutationFn: async ({ transaction }) => { | ||
| await api.save(transaction.mutations) | ||
| }, | ||
| strategy: debounceStrategy({ wait: 1000 }), | ||
| }) | ||
|
|
||
| // Usage in component | ||
| const handleChange = async (value) => { | ||
| const tx = mutate(() => { | ||
| collection.update(id, (draft) => { | ||
| draft.value = value | ||
| }) | ||
| }) | ||
|
|
||
| // Optional: await persistence or handle errors | ||
| try { | ||
| await tx.isPersisted.promise | ||
| } catch (error) { | ||
| console.error("Update failed:", error) | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## Example: Slider with Different Strategies | ||
|
|
||
| ```typescript | ||
| // Debounce - wait for user to stop moving slider | ||
| const mutate = useSerializedTransaction({ | ||
| mutationFn: async ({ transaction }) => { | ||
| await api.updateVolume(transaction.mutations) | ||
| }, | ||
| strategy: debounceStrategy({ wait: 500 }), | ||
| }) | ||
|
|
||
| // Throttle - update every 200ms while sliding | ||
| const mutate = useSerializedTransaction({ | ||
| mutationFn: async ({ transaction }) => { | ||
| await api.updateVolume(transaction.mutations) | ||
| }, | ||
| strategy: throttleStrategy({ wait: 200 }), | ||
| }) | ||
|
|
||
| // Debounce with leading/trailing - save first + final value only | ||
| const mutate = useSerializedTransaction({ | ||
| mutationFn: async ({ transaction }) => { | ||
| await api.updateVolume(transaction.mutations) | ||
| }, | ||
| strategy: debounceStrategy({ wait: 0, leading: true, trailing: true }), | ||
| }) | ||
|
|
||
| // Queue - save every change in order (FIFO) | ||
| const mutate = useSerializedTransaction({ | ||
| mutationFn: async ({ transaction }) => { | ||
| await api.updateVolume(transaction.mutations) | ||
| }, | ||
| strategy: queueStrategy({ | ||
| wait: 200, | ||
| addItemsTo: "back", | ||
| getItemsFrom: "front", | ||
| }), | ||
| }) | ||
| ``` | ||
|
|
||
| ## Implementation Steps | ||
|
|
||
| ### Phase 1: Core Package (@tanstack/db) | ||
|
|
||
| 1. Add `@tanstack/pacer` dependency to packages/db/package.json | ||
| 2. Create strategy type definitions in strategies/types.ts | ||
| 3. Implement strategy factories: | ||
| - `debounceStrategy.ts` - wraps Pacer Debouncer | ||
| - `queueStrategy.ts` - wraps Pacer Queuer | ||
| - `throttleStrategy.ts` - wraps Pacer Throttler | ||
| - `batchStrategy.ts` - wraps Pacer Batcher | ||
| 4. Create core `createSerializedTransaction()` function | ||
| 5. Export strategies + core function from packages/db/src/index.ts | ||
|
|
||
| ### Phase 2: Framework Wrappers | ||
|
|
||
| 6. **React** - Create `useSerializedTransaction` using useRef/useEffect/useCallback | ||
| 7. **Solid** - Create `useSerializedTransaction` using createSignal/onCleanup (matches `useLiveQuery` pattern) | ||
| 8. **Svelte** - Create `useSerializedTransaction` using Svelte stores | ||
| 9. **Vue** - Create `useSerializedTransaction` using ref/onUnmounted | ||
| 10. **Angular** - Create `injectSerializedTransaction` using inject/DestroyRef (matches `injectLiveQuery` pattern) | ||
|
|
||
| ### Phase 3: Testing & Documentation | ||
|
|
||
| 11. Write tests for core logic in packages/db | ||
| 12. Write tests for each framework wrapper | ||
| 13. Update README with examples | ||
| 14. Add TypeScript examples to docs | ||
|
|
||
| ## Strategy Type System | ||
|
|
||
| ```typescript | ||
| export type Strategy = | ||
| | DebounceStrategy | ||
| | QueueStrategy | ||
| | ThrottleStrategy | ||
| | BatchStrategy | ||
|
|
||
| interface BaseStrategy<TName extends string = string> { | ||
| _type: TName // Discriminator for type narrowing | ||
| execute: (fn: () => void) => void | Promise<void> | ||
| cleanup: () => void | ||
| } | ||
|
|
||
| export function debounceStrategy(opts: { | ||
| wait: number | ||
| leading?: boolean | ||
| trailing?: boolean | ||
| }): DebounceStrategy | ||
|
|
||
| export function queueStrategy(opts?: { | ||
| wait?: number | ||
| maxSize?: number | ||
| addItemsTo?: "front" | "back" | ||
| getItemsFrom?: "front" | "back" | ||
| }): QueueStrategy | ||
|
|
||
| export function throttleStrategy(opts: { | ||
| wait: number | ||
| leading?: boolean | ||
| trailing?: boolean | ||
| }): ThrottleStrategy | ||
|
|
||
| export function batchStrategy(opts?: { | ||
| maxSize?: number | ||
| wait?: number | ||
| getShouldExecute?: (items: any[]) => boolean | ||
| }): BatchStrategy | ||
| ``` | ||
|
|
||
| ## Technical Implementation Details | ||
|
|
||
| ### Core createSerializedTransaction | ||
|
|
||
| The core function will: | ||
|
|
||
| 1. Accept a strategy and mutationFn | ||
| 2. Create a wrapper around `createTransaction` from existing code | ||
| 3. Use the strategy's `execute()` method to control when transactions are committed | ||
| 4. Return `{ mutate, cleanup }` where: | ||
| - `mutate(callback): Transaction` - executes mutations according to strategy and returns the Transaction object | ||
| - `cleanup()` - cleans up strategy resources | ||
|
|
||
| **Important:** The `mutate()` function returns a `Transaction` object so callers can: | ||
|
|
||
| - Await `transaction.isPersisted.promise` to know when persistence completes | ||
| - Handle errors via try/catch or `.catch()` | ||
| - Access transaction state and metadata | ||
|
|
||
| ### Strategy Factories | ||
|
|
||
| Each strategy factory returns an object with: | ||
|
|
||
| - `execute(fn)` - wraps the function with Pacer's utility | ||
| - `cleanup()` - cleans up the Pacer instance | ||
|
|
||
| Example for debounceStrategy: | ||
|
|
||
| ```typescript | ||
| // NOTE: Import path needs validation - Pacer may export from main entry point | ||
| // Likely: import { Debouncer } from '@tanstack/pacer' or similar | ||
| import { Debouncer } from "@tanstack/pacer" // TODO: Validate actual export path | ||
|
|
||
| export function debounceStrategy(opts: { | ||
| wait: number | ||
| leading?: boolean | ||
| trailing?: boolean | ||
| }) { | ||
| const debouncer = new Debouncer(opts) | ||
|
|
||
| return { | ||
| _type: "debounce" as const, | ||
| execute: (fn: () => void) => { | ||
| debouncer.execute(fn) | ||
| }, | ||
| cleanup: () => { | ||
| debouncer.cancel() | ||
| }, | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### React Hook Implementation | ||
|
|
||
| ```typescript | ||
| export function useSerializedTransaction(config) { | ||
| // Include strategy in dependencies to handle strategy changes | ||
| const { mutate, cleanup } = useMemo(() => { | ||
| return createSerializedTransaction(config) | ||
| }, [config.mutationFn, config.metadata, config.strategy]) | ||
|
|
||
| // Cleanup on unmount or when dependencies change | ||
| useEffect(() => { | ||
| return () => cleanup() | ||
| }, [cleanup]) | ||
|
|
||
| // Use useCallback to provide stable reference | ||
| const stableMutate = useCallback(mutate, [mutate]) | ||
|
|
||
| return stableMutate | ||
| } | ||
| ``` | ||
|
|
||
| **Key fixes:** | ||
|
|
||
| - Include `config.strategy` in `useMemo` dependencies to handle strategy changes | ||
| - Properly cleanup when strategy changes (via useEffect cleanup) | ||
| - Return stable callback reference via `useCallback` | ||
|
|
||
| ## Benefits | ||
|
|
||
| - β Leverages battle-tested TanStack Pacer utilities | ||
| - β Reduces backend write contention | ||
| - β Framework-agnostic core promotes consistency | ||
| - β Type-safe, composable API | ||
| - β Aligns with TanStack ecosystem patterns | ||
| - β Supports all 5 framework integrations | ||
| - β Simple, declarative API for users | ||
| - β Easy to add custom strategies | ||
|
|
||
| ## Open Questions | ||
|
|
||
| 1. Should we support custom strategies? (i.e., users passing their own strategy objects) | ||
| 2. Do we need lifecycle callbacks like `onSuccess`, `onError` for each mutate call? | ||
| 3. Should batching strategy automatically merge mutations or keep them separate? | ||
| 4. Rate limiting strategy - useful or skip for now? | ||
|
|
||
| ## Notes | ||
|
|
||
| - β Dropped merge strategy for now (more complex to design, less clear use case) | ||
| - The pattern follows existing TanStack patterns where core is framework-agnostic | ||
| - Similar to how `useLiveQuery` wraps core query logic per framework |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.