Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
b68b30f
Adding support for DynamoDB local
gadelkareem Nov 18, 2014
427be85
Documenting DynamoDB support
gadelkareem Nov 18, 2014
da84746
Merge branch 'DynamoDB-local-support'
gadelkareem Nov 18, 2014
cb6bfd5
Code Formatting ( no changes )
gadelkareem Nov 26, 2014
a326ea5
Revert "Code Formatting ( no changes )"
gadelkareem Nov 26, 2014
399d0de
Code Formatting ( no changes )
gadelkareem Nov 26, 2014
3b73b9d
Adding support for find where
gadelkareem Nov 26, 2014
c459afd
Updating repo URL
gadelkareem Nov 26, 2014
7623f82
Fix search with key value
gadelkareem Nov 26, 2014
4b0da53
Fix search with key value
gadelkareem Nov 26, 2014
22aec44
Pagination support
gadelkareem Nov 26, 2014
537175b
Adding sort support
gadelkareem Nov 26, 2014
9382527
Fixing array value search
gadelkareem Nov 27, 2014
64f3919
Error on none matching values on find
gadelkareem Nov 27, 2014
f42f3f2
Removing primaryKeys search based on limit and merging it to scan ins…
gadelkareem Dec 1, 2014
f1f83a1
Fixing limit with loadall
gadelkareem Dec 2, 2014
da105f2
Moving credentials to connection options
gadelkareem Dec 4, 2014
453c388
Removing credentials.json and updating readme file
gadelkareem Dec 5, 2014
602f1d6
Adding npm package
gadelkareem Dec 5, 2014
29fc259
updating for npm
gadelkareem Dec 5, 2014
6407e9b
Adding query using global index
gadelkareem Dec 9, 2014
9f76339
Fixing primary key and index search
gadelkareem Dec 10, 2014
9d2686c
Avoiding querying PK by array
gadelkareem Dec 12, 2014
de1d198
Fix readme
gadelkareem Dec 12, 2014
f8edc0c
Adding not equal comparison operator
gadelkareem Dec 12, 2014
74c6220
- Fixing error on populate
gadelkareem Dec 13, 2014
a4cdf07
clean up
gadelkareem Dec 15, 2014
a1014c7
fixing vogel new api issue
gadelkareem Jan 13, 2015
8d36bfa
remove references to sails for use with waterline standalone.
devinivy Jan 21, 2015
eacc276
Merge pull request #1 from devinivy/remove-sails
gadelkareem Jan 21, 2015
40f7c78
missed one `sails`
devinivy Jan 21, 2015
8e2092d
Merge pull request #1 from devinivy/remove-sails
devinivy Jan 21, 2015
1ef04fd
Merge pull request #2 from devinivy/remove-sails
gadelkareem Jan 21, 2015
61b7280
filter on non-range parameters when not scanning.
devinivy Jan 21, 2015
e13f385
refactor to cache vogels models and support various indices.
devinivy Jan 22, 2015
d3bd29e
fix typos from last commit.
devinivy Jan 22, 2015
f8ae627
user primary keys for update
devinivy Jan 22, 2015
6e6e99a
update update method. deal with update conditionals.
devinivy Jan 23, 2015
666025f
allow identity name to match table name by removing 's'.
devinivy Jan 23, 2015
d4e7514
use column name properly when determining index name.
devinivy Feb 3, 2015
8850d23
Update readme with info on indexes and update.
devinivy Feb 10, 2015
094184d
adjust list so it works with code blocks.
devinivy Feb 10, 2015
06a556a
add to readme, secondary indexes are suffixed
devinivy Feb 10, 2015
97d8410
do not use any index name when just selecting on primary key
devinivy Feb 10, 2015
cca45ad
Add autoPk support by making autoIncrement Vogels' UUID and setting p…
devinivy Feb 12, 2015
a104468
Merge pull request #3 from devinivy/master
gadelkareem Mar 11, 2015
92f5ab5
bump v0.12.0 and adding dohzoh to npm package
gadelkareem Mar 24, 2015
90a3b91
Update index.js
voravor Jun 1, 2015
b799577
Merge pull request #10 from voravor/master
devinivy Jun 2, 2015
9ca0c93
Allow for passing a logger object into AWS from connection.js; tested…
voravor Jun 10, 2015
9fb731a
Merge pull request #11 from voravor/master
devinivy Jun 12, 2015
6592738
0.12.1
devinivy Jun 30, 2015
ffbf0ac
Missing , in the adapter
boadude Oct 29, 2015
06f8053
Merge pull request #18 from boadude/patch-1
devinivy Oct 29, 2015
d6998d5
Fixed missing lodash dependency version
phillip-hall Jan 24, 2016
bff4581
Merge pull request #22 from talvor/master
devinivy Jan 25, 2016
08a82af
Specify async version
devinivy Jan 29, 2016
fa295c1
0.12.2
devinivy Jan 29, 2016
14150f1
added new method to make where queries consistent with mongodb adapter
mattmccartyllc Feb 2, 2016
541e0f9
added support for conditional operators
mattmccartyllc Feb 9, 2016
4c53c2d
added support for conditional operators
mattmccartyllc Feb 9, 2016
e07a78e
fixed findOne() queries based on aws docs
mattmccartyllc Feb 10, 2016
f508448
Merge pull request #23 from mattmccarty/feature/where-query-consisten…
devinivy May 5, 2016
e3de07e
Update README.md for sort info (#36)
ferrants Feb 16, 2017
339f9a4
Creates tables at startup (#38)
liftyourgame Mar 8, 2017
c07817b
increment minor version
ferrants Mar 8, 2017
28ca5e1
Update README.md
ferrants Mar 30, 2017
0cbaa43
Fix startKey usage and support fields with primary and GSI indexes (#40)
ferrants Mar 30, 2017
101697e
Multiple GSI Support (#42)
ferrants May 11, 2017
719e014
Punctuation fixes in documentation. Added Select option (#45)
ShubhankarS Jul 24, 2017
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
165 changes: 161 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,10 +11,9 @@ Install is through NPM.

```bash
$ sails new project && cd project
$ npm install git://github.com/dohzoh/sails-dynamodb.git
$ cp node_modules/sails-dynamodb/credentials.example.json ./credentials.json # & put your amazon keys
$ npm install sails-dynamodb --save
```
Todo: to npm package
Add your amazon keys to your adapter config


## Configuration
Expand All @@ -32,7 +31,11 @@ module.exports.adapters = {
},

dynamoDb: {
adapter: "sails-dynamodb"
adapter: "sails-dynamodb",
accessKeyId: process.env.DYNAMO_ACCESS_KEY_ID,
secretAccessKey: process.env.DYNAMO_SECRET_ACCESS_KEY,
region: "us-west-1",
endPoint: "http://localhost:8000", // Optional: add for DynamoDB local
},

};
Expand All @@ -49,6 +52,160 @@ module.exports.adapters = {
};
```

## Find
Support for where is added as following:
```
?where={"name":{"null":true}}
?where={"name":{"notNull":true}}
?where={"name":{"equals":"firstName lastName"}}
?where={"name":{"ne":"firstName lastName"}}
?where={"name":{"lte":"firstName lastName"}}
?where={"name":{"lt":"firstName lastName"}}
?where={"name":{"gte":"firstName lastName"}}
?where={"name":{"gt":"firstName lastName"}}
?where={"name":{"contains":"firstName lastName"}}
?where={"name":{"contains":"firstName lastName"}}
?where={"name":{"beginsWith":"firstName"}}
?where={"name":{"in":["firstName lastName", "another name"]}}
?where={"name":{"between":["firstName", "lastName"]}}
```
You can specify what attributes/keys should be returned from the query as following:
```
//This will return only name and age in the result (if the field exists in the result)
?where={"name":{"equals":"firstName lastName"}, "select": ["name","age"]}
```

### Pagination
__NOTE__: `skip` is not supported!

Support for Pagination is done using DynamoDB's `LastEvaluatedKey` and passing that to `ExclusiveStartKey`.
See: [DynamoDB Documentation](http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_Query.html)

1. First add a limit to current request

```
/user?limit=2
```

2. Then get the last primaryKey value and send it as startKey in the next request

```
/user?limit=2&startKey={"PrimaryKey": "2"}
```

For more complex queries, you must provide all the fields that are used for an index of the last object returned. An example not using the blueprint apis below. Assume there is a GSI on email (hash) and loginDate (range).

```
// Looking for recent logins by a specific email address
UserLogins.find({
where: {
email: '[email protected]'
loginDate: {"lt": new Date().toISOString()}
},
limit: 10
}).exec((err, userLogins) => {
UserLogins.find({
where: {
email: '[email protected]'
loginDate: {"lt": new Date().toISOString()},
startKey: {
email: '[email protected]',
loginDate: userLogins[userLogins.length - 1].loginDate
}
},
limit: 10
}).exec((err, moreUserLogins) => {
doSomethingWithLogins(userLogins.concat(moreUserLogins));
});
});
```

See that the startKey is in the `where` block and that it has both fields of the Global Secondary Index.

## Using DynamoDB Indexes
Primary hash/range keys, local secondary indexes, and global secondary indexes are currently supported by this adapter, but their usage is always inferred from query conditions–`Model.find` will attempt to use the most optimal index using the following precedence:
```
Primary hash and range > primary hash and secondary range > global secondary hash and range
> primary hash > global secondary hash > no index/primary
```
If an index is being used and there are additional query conditions, then results are compiled using DynamoDB's result filtering. If no index can be used for a query, then the adapter will perform a scan on the table for results.

### Adding Indexes
#### Primary hash and primary range
```
UserId: {
type: 'integer',
primaryKey: 'hash'
},
GameTitle: {
type: 'string',
primaryKey: 'range'
}
```
#### Secondary range (local secondary index)
The index name used for a local secondary index is the name of the field suffixed by "Index". In this case the index name is `TimeIndex`.
```
Time: {
type: 'datetime',
index: 'secondary'
}
```
#### Global secondary index
The index name used for a global secondary index is specified in the `index` property before the type of key (`hash` or `range`). In this case the index name is `GameTitleIndex`.
```
GameTitle: {
type: 'string',
index: 'GameTitleIndex-hash'
},
HighScore: {
type: 'integer',
index: 'GameTitleIndex-range'
}
```

#### Fields with multiple indexes
A field can be both the primary and part of a GSI index. Participating in multiple GSI indexes is supported as of v0.12.5.

```
GameTitle: {
type: 'string',
primaryKey: 'hash'
index: 'GameTitleIndex-hash'
}
```

Multiple GSIs:
```
GameTitle: {
type: 'string',
primaryKey: 'hash'
index: ['GameTitleIndex-hash', 'SomeOtherIndex-hash']
}
```

Multiple GSIs and a secondary index:
```
GameTitle: {
type: 'string',
primaryKey: 'hash'
index: ['secondary', 'GameTitleIndex-hash', 'SomeOtherIndex-hash']
}
```

### Sorting By Indexes
Sorting does not look like how it looks with the normal sails database adapters. You can not sort by an arbitrary field, you must sort by a range field in an index. The index is automatically inferred by what you are querying and you can specify a direction to sort the range fields of the used index. Using the GSI defined above, this will query for descending highscores of Super Mario World:
```
GameScores.find({
where: {
GameTitle: "Super Mario World"
},
sort: "-1"
})
```

## Update
The `Model.update` method is currently expected to update exactly one item since DynamoDB only offers an [UpdateItem](http://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_UpdateItem.html) endpoint. A complete primary key must be supplied. Any additional "where" conditions passed to `Model.update` are used to build a conditional expression for the update. Despite the fact the DynamoDB updates only one item, `Model.update` will always return an array of the (one or zero) updated items upon success.

## Testing

Test are written with mocha. Integration tests are handled by the [waterline-adapter-tests](https://github.com/balderdashy/waterline-adapter-tests) project, which tests adapter methods against the latest Waterline API.
Expand Down
14 changes: 14 additions & 0 deletions contributors.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
###### Contributors
[dozo](https://github.com/dohzoh)
<font color="#999">46 Commits</font> / <font color="#6cc644">943++</font> / <font color="#bd3c00"> 517--</font>
<font color="#dedede">38.98%&nbsp;<font color="#dedede">||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||</font><font color="#f4f4f4">||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||</font><br><br>
[Mike McNeil](https://github.com/mikermcneil)
<font color="#999">31 Commits</font> / <font color="#6cc644">1371++</font> / <font color="#bd3c00"> 725--</font>
<font color="#dedede">26.27%&nbsp;<font color="#dedede">|||||||||||||||||||||||||||||||||||||||||||||||</font><font color="#f4f4f4">|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||</font><br><br>
[Waleed Gadelkareem](https://github.com/gadelkareem)
<font color="#999">27 Commits</font> / <font color="#6cc644">1975++</font> / <font color="#bd3c00"> 1842--</font>
<font color="#dedede">22.88%&nbsp;<font color="#dedede">|||||||||||||||||||||||||||||||||||||||||</font><font color="#f4f4f4">|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||</font><br><br>
[devin ivy](https://github.com/devinivy)
<font color="#999">14 Commits</font> / <font color="#6cc644">586++</font> / <font color="#bd3c00"> 245--</font>
<font color="#dedede">11.86%&nbsp;<font color="#dedede">|||||||||||||||||||||</font><font color="#f4f4f4">|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||</font><br><br>
###### [Generated](https://github.com/jakeleboeuf/contributor) on Tue Mar 24 2015 14:49:47 GMT+0000 (UTC)
5 changes: 0 additions & 5 deletions credentials.example.json

This file was deleted.

Loading