Skip to content

Commit ce3bc59

Browse files
authored
feat: Initial merge of Web UI logic (feast-dev#2352)
* Initial merge of Web UI logic Signed-off-by: Danny Chiao <[email protected]> * Initial merge of Web UI logic Signed-off-by: Danny Chiao <[email protected]> * Initial merge of Web UI logic Signed-off-by: Danny Chiao <[email protected]> * Rebase Signed-off-by: Danny Chiao <[email protected]> * Rebase Signed-off-by: Danny Chiao <[email protected]> * test Signed-off-by: Danny Chiao <[email protected]> * test Signed-off-by: Danny Chiao <[email protected]> * test Signed-off-by: Danny Chiao <[email protected]> * add sample image Signed-off-by: Danny Chiao <[email protected]> * add sample image Signed-off-by: Danny Chiao <[email protected]> * remove comment Signed-off-by: Danny Chiao <[email protected]> * remove comment Signed-off-by: Danny Chiao <[email protected]>
1 parent 1ba86fb commit ce3bc59

File tree

170 files changed

+24951
-17
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

170 files changed

+24951
-17
lines changed

.gitignore

Lines changed: 24 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,6 @@
11
### Scratch files ###
22
scratch*
33

4-
### Feast UI deprecated folder ####
5-
ui/
6-
74
### Local Environment ###
85
*local*.env
96

@@ -197,4 +194,27 @@ sdk/go/protos/
197194
**/registry.db
198195
**/*.aof
199196
**/*.rdb
200-
**/nodes.conf
197+
**/nodes.conf
198+
199+
200+
# Feast UI dependencies
201+
ui/node_modules
202+
ui/.pnp
203+
ui/.pnp.js
204+
ui/coverage
205+
ui/build
206+
ui/feature_repo/data/online.db
207+
ui/feature_repo/registry.db
208+
ui/.vercel
209+
210+
# misc
211+
.DS_Store
212+
.env.local
213+
.env.development.local
214+
.env.test.local
215+
.env.production.local
216+
217+
**/npm-debug.log*
218+
**/yarn-debug.log*
219+
**/yarn-error.log*
220+

README.md

Lines changed: 10 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,11 @@ cd my_feature_repo
4545
feast apply
4646
```
4747

48-
### 4. Build a training dataset
48+
### 4. Explore your data in the web UI (experimental)
49+
50+
![Web UI](ui/sample.png)
51+
52+
### 5. Build a training dataset
4953
```python
5054
from feast import FeatureStore
5155
import pandas as pd
@@ -86,7 +90,7 @@ print(training_df.head())
8690
8791
```
8892

89-
### 5. Load feature values into your online store
93+
### 6. Load feature values into your online store
9094
```commandline
9195
CURRENT_TIME=$(date -u +"%Y-%m-%dT%H:%M:%S")
9296
feast materialize-incremental $CURRENT_TIME
@@ -96,7 +100,7 @@ feast materialize-incremental $CURRENT_TIME
96100
Materializing feature view driver_hourly_stats from 2021-04-14 to 2021-04-15 done!
97101
```
98102

99-
### 6. Read online features at low latency
103+
### 7. Read online features at low latency
100104
```python
101105
from pprint import pprint
102106
from feast import FeatureStore
@@ -165,7 +169,7 @@ The list below contains the functionality that contributors are planning to deve
165169
* [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
166170
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
167171
* [x] [Custom online store support](https://docs.feast.dev/how-to-guides/adding-support-for-a-new-online-store)
168-
* [ ] Bigtable
172+
* [ ] Bigtable (in progress)
169173
* [ ] Cassandra
170174
* **Streaming**
171175
* [x] [Custom streaming ingestion job support](https://docs.feast.dev/how-to-guides/creating-a-custom-provider)
@@ -192,16 +196,16 @@ The list below contains the functionality that contributors are planning to deve
192196
* [ ] Feature Logging (for training)
193197
* **Data Quality Management (See [RFC](https://docs.google.com/document/d/110F72d4NTv80p35wDSONxhhPBqWRwbZXG4f9mNEMd98/edit))**
194198
* [x] Data profiling and validation (Great Expectations)
199+
* [ ] Training-serving skew detection (in progress)
195200
* [ ] Metric production
196-
* [ ] Training-serving skew detection
197201
* [ ] Drift detection
198202
* **Feature Discovery and Governance**
199203
* [x] Python SDK for browsing feature registry
200204
* [x] CLI for browsing feature registry
201205
* [x] Model-centric feature tracking (feature services)
202206
* [x] Amundsen integration (see [Feast extractor](https://github.com/amundsen-io/amundsen/blob/main/databuilder/databuilder/extractor/feast_extractor.py))
207+
* [ ] Feast Web UI (in progress)
203208
* [ ] REST API for browsing feature registry
204-
* [ ] Feast Web UI
205209
* [ ] Feature versioning
206210

207211

docs/getting-started/third-party-integrations.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,7 @@ Don't see your offline store or online store of choice here? Check out our guide
4646
* [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
4747
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
4848
* [x] [Custom online store support](https://docs.feast.dev/how-to-guides/adding-support-for-a-new-online-store)
49-
* [ ] Bigtable
49+
* [ ] Bigtable (in progress)
5050
* [ ] Cassandra
5151

5252
### **Deployments**

docs/roadmap.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,7 @@ The list below contains the functionality that contributors are planning to deve
3737
* [x] [Azure Cache for Redis (community plugin)](https://github.com/Azure/feast-azure)
3838
* [x] [Postgres (community plugin)](https://github.com/nossrannug/feast-postgres)
3939
* [x] [Custom online store support](https://docs.feast.dev/how-to-guides/adding-support-for-a-new-online-store)
40-
* [ ] Bigtable
40+
* [ ] Bigtable (in progress)
4141
* [ ] Cassandra
4242
* **Streaming**
4343
* [x] [Custom streaming ingestion job support](https://docs.feast.dev/how-to-guides/creating-a-custom-provider)
@@ -64,14 +64,14 @@ The list below contains the functionality that contributors are planning to deve
6464
* [ ] Feature Logging (for training)
6565
* **Data Quality Management (See [RFC](https://docs.google.com/document/d/110F72d4NTv80p35wDSONxhhPBqWRwbZXG4f9mNEMd98/edit))**
6666
* [x] Data profiling and validation (Great Expectations)
67+
* [ ] Training-serving skew detection (in progress)
6768
* [ ] Metric production
68-
* [ ] Training-serving skew detection
6969
* [ ] Drift detection
7070
* **Feature Discovery and Governance**
7171
* [x] Python SDK for browsing feature registry
7272
* [x] CLI for browsing feature registry
7373
* [x] Model-centric feature tracking (feature services)
7474
* [x] Amundsen integration (see [Feast extractor](https://github.com/amundsen-io/amundsen/blob/main/databuilder/databuilder/extractor/feast_extractor.py))
75+
* [ ] Feast Web UI (in progress)
7576
* [ ] REST API for browsing feature registry
76-
* [ ] Feast Web UI
7777
* [ ] Feature versioning

infra/templates/README.md.jinja2

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,11 @@ cd my_feature_repo
4343
feast apply
4444
```
4545

46-
### 4. Build a training dataset
46+
### 4. Explore your data in the web UI (experimental)
47+
48+
![Web UI](ui/sample.png)
49+
50+
### 5. Build a training dataset
4751
```python
4852
from feast import FeatureStore
4953
import pandas as pd
@@ -84,7 +88,7 @@ print(training_df.head())
8488

8589
```
8690

87-
### 5. Load feature values into your online store
91+
### 6. Load feature values into your online store
8892
```commandline
8993
CURRENT_TIME=$(date -u +"%Y-%m-%dT%H:%M:%S")
9094
feast materialize-incremental $CURRENT_TIME
@@ -94,7 +98,7 @@ feast materialize-incremental $CURRENT_TIME
9498
Materializing feature view driver_hourly_stats from 2021-04-14 to 2021-04-15 done!
9599
```
96100

97-
### 6. Read online features at low latency
101+
### 7. Read online features at low latency
98102
```python
99103
from pprint import pprint
100104
from feast import FeatureStore

ui/README.md

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# [WIP] Feast Web UI
2+
3+
![Sample UI](sample.png)
4+
5+
This project was bootstrapped with [Create React App](https://github.com/facebook/create-react-app).
6+
7+
## Project structure
8+
The Web UI is powered by a JSON registry dump from Feast (running `feast registry-dump`). Running `yarn start` launches a UI
9+
powered by test data.
10+
- `public/` contains assets as well as demo data loaded by the Web UI.
11+
- There is a `projects-list.json` which represents all Feast projects the UI shows.
12+
- There is also a `registry.json` which is the registry dump for the feature repo.
13+
- `feature_repo/` contains a sample Feast repo which generates the `registry.json`
14+
- `src/` contains the Web UI source code. This parses the registry json blob in `src/parsers` to make this data
15+
available for the rest of the UI.
16+
- `src/custom-tabs` includes sample custom tabs. This is a WIP plugin system where users can inject their own tabs and
17+
data to the UI.
18+
19+
20+
## Available Scripts
21+
22+
In the project directory, you can run:
23+
24+
### `yarn start`
25+
26+
Runs the app in the development mode.\
27+
Open [http://localhost:3000](http://localhost:3000) to view it in the browser.
28+
29+
The page will reload if you make edits.\
30+
You will also see any lint errors in the console.
31+
32+
### `yarn test`
33+
34+
Launches the test runner in the interactive watch mode.\
35+
See the section about [running tests](https://facebook.github.io/create-react-app/docs/running-tests) for more information.
36+
37+
### `yarn build`
38+
39+
Builds the app for production to the `build` folder.\
40+
It correctly bundles React in production mode and optimizes the build for the best performance.
41+
42+
The build is minified and the filenames include the hashes.\
43+
Your app is ready to be deployed!
44+
45+
See the section about [deployment](https://facebook.github.io/create-react-app/docs/deployment) for more information.
46+
47+
### `yarn eject`
48+
49+
**Note: this is a one-way operation. Once you `eject`, you can’t go back!**
50+
51+
If you aren’t satisfied with the build tool and configuration choices, you can `eject` at any time. This command will remove the single build dependency from your project.
52+
53+
Instead, it will copy all the configuration files and the transitive dependencies (webpack, Babel, ESLint, etc) right into your project so you have full control over them. All of the commands except `eject` will still work, but they will point to the copied scripts so you can tweak them. At this point you’re on your own.
54+
55+
You don’t have to ever use `eject`. The curated feature set is suitable for small and middle deployments, and you shouldn’t feel obligated to use this feature. However we understand that this tool wouldn’t be useful if you couldn’t customize it when you are ready for it.
56+
57+
## Learn More
58+
59+
You can learn more in the [Create React App documentation](https://facebook.github.io/create-react-app/docs/getting-started).
60+
61+
To learn React, check out the [React documentation](https://reactjs.org/).

ui/feature_repo/README.md

Lines changed: 72 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,72 @@
1+
# Feast repo
2+
3+
## Overview
4+
5+
This pulls from the dataset used in https://github.com/feast-dev/feast-aws-credit-scoring-tutorial but adds metadata for a full set of Feast FCOs.
6+
7+
This also adds an on demand feature view + feature services + a saved dataset.
8+
9+
## Setting up Feast
10+
11+
Install a dev build Feast using pip
12+
13+
Clone a feast repo:
14+
```bash
15+
git clone https://github.com/feast-dev/feast.git
16+
```
17+
18+
Install a dev build of feast
19+
```bash
20+
cd feast
21+
pip install -e "sdk/python[dev]"
22+
```
23+
24+
Then for this demo, you'll actually need to fix a bug by adding this to `type_map.py#L144`:
25+
```python
26+
if isinstance(value, np.bool_):
27+
return ValueType.BOOL
28+
```
29+
30+
## Test features
31+
We have already set up a feature repository here using test data from [data](data)). Features have already been pre-materialized to a local sqlite online store. The results of `feast registry-dump` have been thrown into [registry.json](../public/registry.json)
32+
33+
To query against this registry, you can use run the `test_get_features.py`
34+
```bash
35+
python test_get_features.py
36+
```
37+
38+
Output:
39+
```
40+
--- Historical features (from saved dataset) ---
41+
mortgage_due credit_card_due missed_payments_1y total_wages dob_ssn event_timestamp state tax_returns_filed location_type population city zipcode
42+
0 741165 2944 3 71067272 19781116_7723 2021-04-12 08:12:10+00:00 MI 2424 PRIMARY 4421 WEIDMAN 48893
43+
1 91803 8419 0 534687864 19530219_5179 2021-04-12 10:59:42+00:00 GA 19583 PRIMARY 38542 DALTON 30721
44+
2 1553523 5936 0 226748453 19500806_6783 2021-04-12 15:01:12+00:00 TX 6827 PRIMARY 12902 CLEBURNE 76031
45+
3 976522 833 0 34796963 19931128_5771 2021-04-12 16:40:26+00:00 VA 1287 PRIMARY 2342 GLADE HILL 24092
46+
47+
--- Online features ---
48+
city : ['DALTON']
49+
credit_card_due : [8419]
50+
dob_ssn : ['19530219_5179']
51+
location_type : ['PRIMARY']
52+
missed_payments_1y : [0]
53+
mortgage_due : [91803]
54+
population : [38542]
55+
state : ['GA']
56+
tax_returns_filed : [19583]
57+
total_wages : [534687864]
58+
zipcode : [30721]
59+
city : ['DALTON']
60+
credit_card_due : [8419]
61+
dob_ssn : ['19530219_5179']
62+
location_type : ['PRIMARY']
63+
missed_payments_1y : [0]
64+
mortgage_due : [91803]
65+
population : [38542]
66+
state : ['GA']
67+
tax_returns_filed : [19583]
68+
total_wages : [534687864]
69+
transaction_gt_last_credit_card_due : [False]
70+
zipcode : [30721]
71+
72+
```
21.4 MB
Binary file not shown.

0 commit comments

Comments
 (0)